Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [largedatavolumes]

Questions dealing with handling very large data volumes (greater than 2 Million records in an object) present in large enterprise organizations.

0
votes
1answer
16 views

LDV : What is max records that aggregate query can work with?

I am having difficulty understanding limits on aggregate queries. Salesforce Documentation for ReadOnly Attribute of Visualforce Page says Normally, queries for a single Visualforce page request ...
0
votes
0answers
184 views

Retrieve large data sets from salesforce using Rest API or Bulk API

We have external system which retrieves the data from Salesforce by using Rest Api. It was working fine till the return resultSet is limited. However, when we are working on large data set it works as ...
0
votes
1answer
27 views

Query not selective error is happening only in Trigger context but not in VF,Batch. Is this expected?

I have a query with query plan as below: The lowest cost is greater than 1, It means query is not selective. Trigger context: I used this query in a before insert trigger on a empty object and it ...
4
votes
2answers
94 views

Bulk deletion of Case Team Member records, data skew, and performance issues

We're proposing to delete no-longer-relevant Case Team Member records, numbering about 4.3 million (2.8 million total Cases). I'm concerned about the performance implications that might be caused by ...
0
votes
0answers
22 views

Can campaign members create data skew?

So, basically I've come across a use case where we want to have an average of 50.000 campaign members per campaign, and I'm wondering if this can be the origin of data skew or, by any chance, ...
1
vote
0answers
84 views

How do you avoid a timeout on custom objects with a lot of data?

We have a batch performing a query on a custom object that returns a timeout. The table contains 2.4 million records and the query should return 125 000 records. That's 5% of total records. Our ...
3
votes
1answer
85 views

500K record transaction: Trigger vs Scheduled Apex?

I'm working on an instance where records are sent from Eloqua over to Salesforce in a single transaction of 300,000 to 500,000 records. I didn't find out until NOW. Based on requirements, I've ...
2
votes
0answers
175 views

How to migrate large data volume from On-premise to Heroku Postgres [closed]

We are planning to migrate large volume of data from on-premise Oracle DB to Heroku Postgres in Private Space. We are using Heroku to build our API Services for various channels e.g. portal / mobile ...
5
votes
0answers
150 views

Working with Very Large SOQL Queries

Background This is a question about your experiences managing huge amount of data for UIs. With an ERD like this, where the arrow directions reflect the fields lookup directions. (e.g. entity "B" ...
1
vote
1answer
39 views

How to Filter New Accounts - Large Data Volumes?

My business partners are adding in 1000000 new records into Salesforce and they want to document at a location level as well as keep the current data documentation at an org level untouched with no ...
1
vote
1answer
82 views

Reading large number of records in REST service

This is a apex class for a REST service fro inserting multiple records into a custom salesforce object- I am getting a System.LimitException: Too many query rows: 50001 error because my org data is ...
5
votes
1answer
623 views

Bulk data API: How to Avoid Upserts

I'm working on an daily integration of about 5 million records (file delta's is not an option). Best practices for large data volumes recommends avoiding upserts. The only alternative solution I can ...
1
vote
1answer
58 views

What is the best practice method to summarize contact information to leads?

Here's the business case. I want to have a reportable field on a lead that displays the number of contacts for, say, a city field match. This value could be used in reports, dashboards, snapshots, ...
0
votes
1answer
17 views

High volume imports

I would like to know best practices when dealing with nightly loads of high volume of information, let's say, importing around 2.500.000 regs. per day into CRM. thanks, Daniel
1
vote
1answer
286 views

Rest API PATCH method with Large Data Volumes

Are there any best practices relating to the use of the Rest API within an environment with large data volumes? I am aware of the SOQL best practices relating to LDV. Does this relate at all to ...
0
votes
1answer
66 views

Multiple File Data Set Import Cleaning

It is pre-Salesforce import, I have designed the Named Account structure inside Salesforce, and I have 1500 Hoover's D&B Family Trees with the same columns and headers in .xls form. What do I use ...
0
votes
0answers
18 views

Is there a way to tell if custom indexes exist on an object?

I want to be able to design to use indexes and custom indexes where they exist. Is there a way to tell if custom indexes have previously been created on a field?
2
votes
1answer
215 views

Best approach for creating Junction object with millions of records

I have two objects Object1 and Object2 in Salesforce. These two objects doesn't have Master-Detail relationship. But they are related through a Text column X. I want to create many to many junction ...
1
vote
1answer
1k views

Querying more than 15000 records into a VF page using RemoteAction

I have a page that uses AngularJS for front-end rendering and filtering. To make use of the dynamic filtering capability of AngularJS filters, I am loading all the records onto the HTML page, so the ...
13
votes
2answers
1k views

Can SOSL perform full text search on chatter files?

The Document sobject has an IsBodySearchable flag which controls if you can full text search it via SOSL. ContentVersion has no such flag but I know that it is searchable in certain cases. However, ...
3
votes
1answer
4k views

query against large object type

I have my own custom code(before insert trigger) for check on Dead Leads. for storing dead Leads there is a custom object. Currently the custom object has more than 200000 records. Due to large ...
6
votes
1answer
955 views

Querying Large volumes of data in Apex

I am looking for a solution where i have a scenario to query large volume data and do an update on them. Is there a way i can do, query once and then querymore on that object instead of getting all ...
3
votes
2answers
3k views

Batch Apex throwing error when executing query ( SQLException [common.exception.SfdcSqlException: ORA-01013:)

I am trying to execute a Batch job that is processing around 2 million contacts, I batch Apex whenever I try to execute is throwing an error First error: SQLException [common.exception....
4
votes
1answer
3k views

Reports and formula fields

I read this very helpful article, Designing Dashboards and Reports for Force.com Implementations with Large Data Volumes I have one question regarding formula fields - if I use formula fields instead ...
3
votes
2answers
5k views

“Non-selective query against large object type” only occurs in trigger context despite filtering on lookups

I have a very simple after delete trigger that searches for orphan records and deletes them: List<my_ns__Orphanable_Object__c> orphans = Database.query( ' SELECT ' + ' Id ' + ' ...
4
votes
3answers
400 views

Moving Salesforce data into Heroku Java app for LDV processing?

I have read a few blog posts about moving parts of an app into a Ruby Heroku app, but no example for moving data between a Salesforce org and a custom Java app on Heroku. Do you know of any in posts ...
0
votes
1answer
118 views

How to address large data?

AtaHi guys: i have a things about salesforce account tables. In my scenario, I have more than 600000 record in my account table. And currently, we want to admin all account territory. So, I want to ...
3
votes
2answers
105 views

Prevent LDV-problems in own ISV package by auto-indexing packaged fields

In our managed ISV package we have a custom object that in many customer orgs might run into LDV issues as the nature of this object lead to million of records. I know that Salesforce support can ...
7
votes
3answers
2k views

Unable to perform a COUNT() query on object with many million records

I have a custom object with around 25.000.000 records and need to perform a single count query to find those records where a field has a certain value. I could get none of the following queries to ...
19
votes
2answers
17k views

Breaking out of a nested loop

Just for a bit of background I am trying to write a Depth First Sort in Apex. I have been successful writing the method using recursion but when testing with some large data volumes I have hit the ...
2
votes
1answer
6k views

More than 50000 items inside a list in apex

Can we store more than 500000 items into a list in apex? I tried this and is working fine. So the limit of 50000 has been lift up? When this update was made to salesforce? any idea?
1
vote
1answer
345 views

Heapsize problem on VF page

I am VF page which display records for a custom object. When there are a large number of custom records - I get: HEAPSIZE 6014176 error on page. Any tips?
10
votes
3answers
3k views

Governor Limit question: Large data set. How do I avoid hitting governor limits?

We have a pretty complex trigger suite on our org. We've built a lot of custom functionality. I'm starting to worry about governor limits and not really understanding how to manage them. I've read ...
12
votes
4answers
17k views

How to query more than 50000 records in apex and bind it in vf page in jqgrid and paging?

How to query more than 50000 records (78000 records) in apex and bind it in vf page in jqgrid and implement paging in it? I tried querying through SOQL and assigning it to list.. Am unable to query ...