Friday, 24 November 2017

Big Object - Bring Big Data In Salesforce

Big Object

Hi guys today I am going to discuss about Big Object  in salesforce. Big Object allows you to store tons of data into salesforce platform. It provide consistent performance whether there are 1 million records or 1 Billion records . But we can store data in custom object also then why we use Big Object .

Big object is worth to use if you want to store historical data.

Eg. All the Case record which are closed and were created years ago.
Transaction history of a user.  

There are 2 type of Big Object In salesforce.

  1. Standard Big Object : These Object are defined by Salesforce itself.
  2. Custom Big Object :These Object are defined by user in its Salesforce org.

Custom Object VS Big Object :

Sr no
Custom Object
Big Object
Suffix is __c
Suffix is __b
Can be created from UI.
Can’t be created from UI. We must need  to use metadata api to create Big Object and add field to Custom Big Object
Support standard ui element such as list view , details page.
Do not support standard ui element such as list view , page layout.
Standard Reports and Dashboard are available.
Standard Report and Dashboard are not available.
Limit for maximum no of Custom Object can be created in org. Depend on the license type. In developer Org it is 400.
Limit for maximum no of Big Object can be created in org. Depend on the license type. In developer Org it is 100.
Triggers, flow and process builder are available.
To support  the scale of data in Big Object Triggers, flow and process builder are not available.
Support lot of data types.
Support DateTime, Lookup, Number, Text, and LongText Area only
Custom fields can be deleted.
Custom fields can not be deleted once created.
Records can be updated and deleted.
Records Can’t be deleted once inserted.

How to Define and Deploy Big Object :

To define and deploy Big Object we need to make metadata. After deploying we can view in setup ui under Customize >> Create >> Big Object

Define a Big Object : Define a custom big object through the Metadata API by creating XML files that contain its definition, fields, and index . Files we need are specified below.

  1. Object File :- Create a .object file. The name of file should be similar to API of object. Eg. myobject__b.object
  2. permissionSet/Profile file :- This file is not required. We create permissionSet or profile file to give access permission for each field.
  3. package.xml  :- Create a file for the metadata package to specify the contents.

Let’s see an example. We are creating a Big Object Employee (You can also download source file. Click here)
So 1st we will make an Employee__b.object file . It will look like this.

<?xml version="1.0" encoding="UTF-8"?> <CustomObject xmlns=""> <deploymentStatus>Deployed</deploymentStatus> <fields> <fullName>First_Name__c</fullName> <label>First Name </label> <length>50</length> <type>Text</type> <unique>false</unique> <required>true</required> </fields> <fields> <fullName>Salary__c</fullName> <label>Salary</label> <type>Number</type> <scale>2</scale> <precision>16</precision> <required>false</required> <unique>false</unique> </fields> <fields> <fullName>Dob__c</fullName> <label>Date Of Birth</label> <type>DateTime</type> <required>false</required> <unique>false</unique> </fields> <fields> <fullName>Contact__c</fullName> <label>Contact</label> <referenceTo>Contact</referenceTo> <relationshipName>Employee_Con</relationshipName> <type>Lookup</type> </fields> <indexes> <fullName>EmployeeIndex</fullName> <label> Employee index</label> <fields> <name>First_Name__c</name> <sortDirection>DESC</sortDirection> </fields> </indexes> <label>Employee</label> <pluralLabel>Employee</pluralLabel> </CustomObject>
Now we will make perssionSet File. It’s extension will be .permissionSet

<?xml version="1.0" encoding="UTF-8"?> <PermissionSet xmlns=""> <fieldPermissions> <editable>true</editable> <field>Employee__b.firstName__c</field> <readable>true</readable> </fieldPermissions> <fieldPermissions> <editable>true</editable> <field>Employee__b.email__c</field> <readable>true</readable> </fieldPermissions> <fieldPermissions> <editable>true</editable> <field>Employee__b.salary__c</field> <readable>true</readable> </fieldPermissions> <fieldPermissions> <editable>true</editable> <field>Employee__b.dob__c</field> <readable>true</readable> </fieldPermissions> </PermissionSet>
In last we need package file. (package.xml)

<?xml version="1.0" encoding="UTF-8"?> <Package xmlns=""> <types> <members>*</members> <name>CustomObject</name> </types> <types> <members>*</members> <name>PermissionSet</name> </types> <version>41.0</version> </Package>

Deploy Custom Big Object : We are using workbench to deploy. To create zip (package) file for deploying custom Big Object make sure that the object file should be in objects folder, permissionSet file should be in permissionSets folder and the package.xml file should be in root directory (must not inside any folder).

See the image

Once you have successfully deployed Big object you can view them under Big Object in setup ui .

When we see the details of Employee Big Object we see that there is no button is available for delete and create new custom field . What if we want to add some new fields and want to delete any of existing.
Let me clear this that you can’t delete a custom field created on Big Object.  To add new custom field we will go through metadata deployment.

Add new Custom Field to Custom Big Object :-  Earlier we created  a custom Big Object . Now we want to add a new custom field in it.
We need to create an .object file with the same name of previous one (Employee__b.object). It will contain the following code
<?xml version="1.0" encoding="UTF-8"?> <CustomObject xmlns=""> <deploymentStatus>Deployed</deploymentStatus> <fields> <fullName>Last_Name__c</fullName> <label>Last Name </label> <length>50</length> <type>Text</type> <unique>false</unique> </fields> <label>Employee</label> <pluralLabel>Employee</pluralLabel> </CustomObject>
And now we will deploy it using workbench. For deploying we will follow the previous example structure . This time we are not using permissionSet file . package.xml will be same as previous. After deployment we can see there is new Custom  field Last Name on Employee__b Big Object.  

Insert Data into Big Object :
Using CSV :- To insert data into Big Data we can upload csv file using data loader.

Using Apex Code :-  Using Database.insertImmediate() we can insert data into Big Object.
Here is the snippet using  it you can insert data into our Employee Object.
List<Employee__b> listOfEmp = new List<Employee__b>(); Employee__b objEmp = new Employee__b(); objEmp.first_Name__c = 'Naveen'; objEmp.Salary__c = 5000; objEmp.Last_Name__c = 'Soni'; listOfEmp.add(objEmp); Database.insertImmediate(listOfEmp);

Note : Big objects don’t support transactions including both big objects, standard object and        custom objects
Then How can we Insert records in Big Object. Solution is “Async SOQL”, we will go through it in next steps.
If you are not able to insert record in then check for field permission for your profile. Even if you are a system administrator .

Update  Data into Big Object :  

Code snippet :

List<iarpit__Employee__b> listOfEmp = new List<iarpit__Employee__b>(); for(iarpit__Employee__b objEmp : [SELECT iarpit__First_Name__c, iarpit__Last_Name__c FROM iarpit__Employee__b WHERE iarpit__First_Name__c='Ankit']){ iarpit__Employee__b objEmp1 = new iarpit__Employee__b(); objEmp1.iarpit__last_name__c = 'rohit'; objEmp1.iarpit__first_name__c = objEmp.iarpit__first_name__c; listOfEmp.add(objEmp1); } System.debug(listOfEmp); if(listOfEmp.size() > 0){ Database.insertImmediate(listOfEmp); }

Want to view records of Big Object :-

As I already mention that there is no standard ui is available for Big Object. So if we want to see data then we can create our custom visualforce page and view data on it.

Here I created a vf page which display first name and last name of employee.


public class BigObjExample{ public List<Employee__b> listOfEmp {get;set;} public BigObjExample(){ listOfEmp = [SELECT iarpit__First_Name__c, iarpit__Last_Name__c FROM iarpit__Employee__b ]; if(listOfEmp == null){ listOfEmp = new List<Employee__b>(); } } }
<apex:page controller="BigObjExample" title="Big Object Example"> <apex:sectionHeader title="Employee" subtitle="All Employee"/> <apex:pageBlock> <apex:pageBlockSection columns="1"> <apex:pageBlockTable value="{!listOfEmp}" var="emp"> <apex:column value="{!emp.first_name__c}"/> <apex:column value="{!emp.last_name__c}"/> </apex:pageBlockTable> </apex:pageBlockSection> </apex:pageBlock> </apex:page>

Screenshot ;

SOQL In Big Object  :-
There are some limitation for soql query on Big Object. We can use all the fields in ‘WHERE’ , all operators are not supported.
Only the indexed fields can be used in WHERE condition in soql.
You can use =, <, >, <=, or >=, or IN on the last field in your query. Any prior fields in your query can only use the = operator. The !=, LIKE, NOT IN, EXCLUDES, and INCLUDES operators are not valid in any query
Async SOQL :  
When we want to query large amount of data we go for  Async SOQL .
Async SOQL queries are run in the background over Salesforce entity data, standard objects, custom objects, and big objects.
Async SOQL is implemented as a RESTful API that enables you to run queries in the familiar syntax of SOQL.
Result of each Async SOQL query are being stored in another object which you define. It can be a Standard , Custom or Big Object.
Note :Async SOQL for standard and custom objects are not generally available.
URL on which we send request for Async SOQL
We set query in the body of HttpRequest.
Example of Query :-
{ "query": "SELECT iarpit__First_Name__c FROM iarpit__Employee__b", "operation": "insert", "targetObject": "iarpit__BigEmp__c", "targetFieldMap": {"iarpit__First_Name__c":"name"} }
In this I have created a Custom Object BigEmp__c
Snippet of class using we can send request for Async SOQL :-
/* Name : BigObjectToCustom Date : 07/11/2017 Author : Arpit Vijayvergiya Description : this is being used for transfering data from Big object to Custom object */ public class BigObjectToCustom{ public BigObjectToCustom(){ String query = '{"query": "SELECT iarpit__First_Name__c FROM iarpit__Employee__b","operation": "insert","targetObject": "iarpit__BigEmp__c","targetFieldMap": {"iarpit__First_Name__c":"name"}}'; Http http = new Http(); HttpRequest request = new HttpRequest(); request.setMethod('POST'); request.setEndPoint(''); request.setHeader('Content-Type', 'application/json'); request.setBody(query); request.setHeader('Authorization', 'Bearer ' + UserInfo.getSessionID()); HttpResponse response = http.send(request); System.debug(response.getBody()); } }
Note : As per the winter 18 release doc “Async SOQL is included only with the licensing of additional big object capacity.

If we are not having additional permission then we will be getting this message in response.

“ [{"message":"This feature is not currently enabled for this user type or org: [AsyncQuery]","errorCode":"FUNCTIONALITY_NOT_ENABLED"}] ”



  1. well explained blog arpit. keep it up..
    waiting for your next blog

    1. Big data is a term that describes the large volume of data – both structured and unstructured – that inundates a business on a day-to-day basis. big data projects for students But it’s not the amount of data that’s important.Project Center in Chennai

      Spring Framework has already made serious inroads as an integrated technology stack for building user-facing applications. Corporate TRaining Spring Framework the authors explore the idea of using Java in Big Data platforms.

      Spring Training in Chennai

      The new Angular TRaining will lay the foundation you need to specialise in Single Page Application developer. Angular Training

  2. This is good and helping to understand big object. Right now there is less amount of article are there. Keep it up. You are doing very good.

  3. Very helpful blog ...waiting for another these type of blogs from ur side.

  4. Very helpful blog ...waiting for another these type of blogs from ur side.

  5. Nice blog. One thing that you can definitely update records in big object but you can't delete them. Insertimmediate will do the update operation as well if we map the index fields correctly.

  6. Hi,
    I'm getting an Error while executing Async SOQL as "errorCode":"INSUFFICIENT_ACCESS","message":"You don’t have permission to query this object Account with Async SOQL.
    What Permissions I have to give ,Am I Missing anything here ...Please Let Me Know Thanks in Advance

  7. @Mubeena - Make sure you have big object permission..Go to your profile and permission set and check for the access.

    1. This comment has been removed by the author.

    2. @Prashant Pandey I checked for profile and permission set but no use i'm getting same error "errorCode":"INSUFFICIENT_ACCESS","message":"You don’t have permission to query this object Account with Async SOQL. while inserting records from standard or custom object into bigobject.

  8. Very Nice Blog and very well explained.

  9. Hi Arpit,

    I am facing problem: No package.xml found. can you tell my why I am facing it.
    If I download your file it is working fine. but If I unzip it and again zip same file then getting problem: No package.xml found error.
    can you help me to overcome to it.


  10. I get the error "Path must start with /" when I use the custom URL Link When I use the /services/data/v41.0/async-queries/ URL I get the 'Insufficient_Access" error message even though I am the System Admin and I have checked to make sure that I have full permission on both the Custom_Object__c and Big_Object__b.

  11. Hi Arpit,

    I have done all the steps mentioned above and can see data inside big objects through Visualforce Page. But after uploading more than 1000 records, it gives me error as "Collection size 1,486 exceeds maximum size of 1,000" .I have overcome this error before by introducing pagination for vf pages, but bog object does not support Offset, so what can be used to correct this issue?


  12. Thank you for sharing your article. Great efforts put it to find the list of articles which is very useful to know, Definitely will share the same to other forums.

    java training in chennai

  13. Very good information. Its very useful for me. We need learn from real time examples and for this we choose good training institute, we need to learn from experts . So we make use of demo classes . Recently we tried Big Data demo class of Apponix Technologies.

  14. how to resolve this error : -
    “ [{"message":"This feature is not currently enabled for this user type or org: [AsyncQuery]","errorCode":"FUNCTIONALITY_NOT_ENABLED"}] ”

  15. Is there any way to deploy zip(package) file through apex.

  16. Thanks for sharing useful information on big data technology. Bigdata is going to be future of the computing world in the coming years. This field is a very good option that provides huge offers as career prospects for beginners and talented professionals. So, taking bigdata training institute in bangalore will help you to start good career in big data technology.

  17. Thank you for sharing the very useful information. It has helped me a lot. Thank you

  18. I am definitely enjoying your website. You definitely have some great insight and great stories. for more info

  19. Thanks for the blog loaded with so many information. Stopping by your blog helped me to get what I was looking for. here

  20. Web Host Review - After being shut down without warning, you will learn how to review your web hosting provider before handing over your credit card. top web hosts in 2020

  21. This deplorable circumstance happens on the grounds that the Forex market is ever evolving. Indeed, even specialists can't foresee the development with any extraordinary level of conviction.simultaneous localization and mapping

  22. Technology alludes the information and usage of devices, methods and frameworks to fill a greater need like tackling issues or making life simpler and better. Its criticalness on people is huge on the grounds that technology causes them adjust to the climate. nest headsets for work from home

  23. You make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. Plombier Paris

  24. I simply wanted to write down a quick word to say thanks to you for those wonderful tips and hints you are showing on this site. As a result of checking through the net and meeting techniques that were not productive, Same as your blog I found another one Oracle APEX .Actually I was looking for the same information on internet for Oracle APEX and came across your blog. I am impressed by the information that you have on this blog. Thanks once more for all the details.

  25. it's really cool blog. Linking is very useful have really helped Business

  26. Great article Lot's of information to Read...Great Man Keep Posting and update to People..Thanks servientrega centro de soluciones

  27. The web site is lovingly serviced and saved as much as date. So it should be, thanks for sharing this with us. Business

  28. Reducing IT expenses and overseeing ITBizTek foundation are just important for the condition.

  29. Acknowledges for paper such a beneficial composition, I stumbled beside your blog besides decipher a limited announce. I want your technique of inscription... get a list of the best 5 movers and packers

  30. You may have the technology to fabricate a superior mousetrap, however on the off chance that you have no mice or the old mousetrap functions admirably, there is no chance and afterward the technology to assemble a superior one gets unessential. Then again, on the off chance that you are overwhelmed with mice, at that point the chance exists to improve an item utilizing your technology. remote team todo list

  31. HDMI is the standard interface for the new top quality TVs and home theater frameworks coming out today. A solitary HDMI link is comprised of an aggregate of 19 separate wires joined into one.PS5 Spare Parts

  32. Some small businesses often hold direct meetings with selected customers in order to get feedback. Salesforce consulting can help you in determining if this will be a useful idea for your business. Salesforce interview questions

  33. I can set up my new idea from this post. It gives in depth information. Thanks for this valuable information for all,.. I can set up my new idea from this post. It gives in depth information. Thanks for this valuable information for all,..

  34. If you have been overwhelmed by the enormous cost of electricity, there are steps that can be taken to cut down on electricity costs. These solutions are simple to implement, and will save you money on your electricity bill.

  35. A Non-Disclosure Agreement (NDA) will be marked or Corporate and Business Clients to guarantee privacy of information from accommodation of media to information conveyance.advice on data recovery

  36. I have read your excellent post. This is a great job. here you can iesco bills
    IESCO Bill Calculator
    IESCO Duplicate Bills

  37. Something that is vital to our military today. Harmony through strength I generally say. Some say utilizing these mechanical devices isn't battling reasonable - right - why battle reasonable - battle to win. Hello, nobody likes war, however in the event that you end up drenched in one, you need to win it.simultaneously localizing the robot and mapping with qslam

  38. This is important, though it's necessary to help you head over to it weblink: email contacts extractor

  39. This is often great. Person watch this finding video and we are confused. We are attracted to one of these details
    cell phone holder for car
    garlic press

  40. Thanks for sharing such a amazing Article! It's really Helpful.

  41. This Article is Really Helpful for Everyone! Thanks for sharing!

  42. Creative Web Studio - The Cyber Defense Company bietet als zertifiziertes Unternehmen lösungsorientierte und zeitgemässe ICT-Services für KMUs an Hauptfokus: Cloud, IT-Security und

  43. This is just the beginning, as this new law starts trends in the State of Nevada, and different states will glance towards it in concocting their own transit regulations with respect to automated vehicles. Without a doubt I trust you will kindly think about this and think on it.this website

  44. Thanks for Sharing, Very informative post! Also Visit

  45. I like what you guys are usually up too. Such clever work and reporting!
    Keep up the amazing works guys I’ve included
    you guys to blogroll.

  46. With special privileges and services, UEFA BET offers opportunities for small capitalists. Together ufa with the best websites that collect the most games With a minimum deposit starting from just 100 baht, you are ready to enjoy the fun with a complete range of betting that is available within the website

    ufabet , our one another option We are a direct website, not through an agent, where customers can have great confidence without deception The best of online betting sites is that our Ufa will give you the best price

  47. Obviously, before this happens we need some normalization and every one of the vehicles should be taking to one another while minding out your solicitation for its expected objective - the vast majority of those chipping away at these advances concur with this reality.SLAM algorithm

  48. There are a few manners by which your PC can get contaminated with a rootkit. They can come enveloped by email connections or packaged with programs you download. You can get tainted just by visiting a malignant site. Rootkits can likewise be stacked from a circle or USB drive by a transgressor who acquires admittance to your PC for only a couple minutes.scan for rootkits

  49. I’m gone to inform my little brother, that he should
    also visit this web site on regular basis to take updated from most up-to-date news update.

    skills you can learn online

  50. It developed to make organizations function efficiently and profitably by reducing the cost of managing hardware infrastructure. Salesforce training gets a good popularity ensuring that the company performs well in long run. salesforce interview questions

  51. You can found a professional institute who are famous for Salesforce Online Training in Ameerpet, Hyderabad, India. Are you really interested to make your career as Salesforce Consultant? Then take a chance to interact with us, our Salesforce Online Training will surely make a difference. Salesforce training in Hyderabad

  52. I really liked your blog article. Really looking forward to read more. Awesome.guest internet hotspot gateway

  53. Thank you so much for sharing this amazing post.

  54. In these difficult circumstances, robots are needed to gather data about their environmental factors to stay away from hindrances. To dive more deeply into the key and basic components of versatile robots, read on. object dictionary patent

  55. Robots can improve on any cycle. They can attempt any activity without relying upon anything. They have a considerable rundown of utilizations where they can help mankind. patented tech used on astro

  56. También estoy buscando la misma solución para mi nuevo blog del sitio web de la herramienta de facturación.
    fesco online bill

  57. explore sora solutions services, a Swiss-based Information Technology company. Learn more about our complete hardware and software solutions.

  58. Many studies have shown a double-digit rise in worldwide ecommerce sales in 2016. Many analysts even predict that the worldwide ecommerce sales will increase consistently till 2020. However, the number of people accessing ecommerce websites on their mobile devices has been increasing rapidly. competitors pricing

  59. Alex Hassen Says: Thank you so much sharing this article.

  60. We have sell some products of different custom is very useful and very low price please visits this site thanks and please share this post with your friends.

  61. When you use a genuine service, you will be able to provide instructions, share materials and choose the formatting style. Norstar

  62. I'm extremely inspired along with your writing skills as neatly as with the format in your blog. Is that a paid subject or did you customize it yourself? Anyway stay up the excellent quality writing, it's rare to look at a great blog like this one today.. Bathroom Mirror

  63. Thanks for your marvelous posting! I really enjoyed reading it, you could be a great author. I will be sure to bookmark your blog and definitely will come back in the foreseeable future. I want to encourage you to continue your great posts, have a nice afternoon! Digital Marketing Course with Placement