1

I wrote a powershell script a few years to extract all my Classic Knowledge Articles into a JSON file (one JSON file per Article Type). It worked very well - and I was then able to use powershell to chunk the export into smaller files with only 200 records so I could reimport them.

After not using this script for a year or two, I just tried running it again, and found that sfdx was only exporting 200 records.

The line in my script to perform the export is like so:

sfdx force:data:tree:export -q "$query" -d $config.sourceOrg --targetusername $config.sourceOrg

The query it uses is dynamically generated - but the query is fine as when I run it in dev console I get the full number of articles returned.

Has the behaviour for SFDX Tree Export changed? And why is it only exporting 200 records? I see that it is supposed to have a maximum of 2000 records, not 200.

6
  • IN VS Code terminal I ran a tree export with a query which I knew would get a lot of records, I got this: WARNING: Query returned more than 200 records. Please run the command using the plan option instead. Record Count: 2000 But I also got this: Wrote 2000 records to data\Account.json. I verified that there are 2000 records in the JSON file.
    – Moonpie
    May 11 '21 at 0:02
  • My query: sfdx force:data:tree:export -q "SELECT Name FROM Account" -d .\data\ -u testval. I do not know whether it is relevant to your issue, but your -d option appears to be an org rather than a folder/directory/path where your JSON file will be written. The -d option = --outputdir = "directory to store files". Am I reading your command correctly?
    – Moonpie
    May 11 '21 at 0:13
  • Thanks Moonpie - that is great to see yours ran correctly - so must be my error. You did read my query right - but that is just my naming convention. I save the data into a folder with the same name as the org I got the data from.
    – bpilling
    May 11 '21 at 0:20
  • I tried two more exports in order to try to rule out/in anything. My original sample query (posted in previous comment) was in a sandbox, so just to help verify that it is not a production org limitation, I ran it in production: I got the same warning and yet 2000 records.
    – Moonpie
    May 11 '21 at 12:14
  • In order to rule out/in that it was an object issue, I ran sfdx force:data:tree:export -q "SELECT ArticleNumber FROM KnowledgeArticle" -d .\data\ -u myProdOrg". We only have 258 articles, but at least it is over 200. I received the same warning, but all 258 articles ended up in the JSON.
    – Moonpie
    May 11 '21 at 12:16
4

You need to use --plan to get more records out. If you don't have a plan, you're technically limited to the REST API that powers importing. It's limited to 200 records across all trees, so to import more than 200 records, you need a plan file, and the files will be imported in batches of 200.

10
  • Thanks sfdcfox. Do you know the answer to other part of my question - is this new or changed behaviour? I didn't seem to need to do this last time I ran this script (well over a year ago at least ).
    – bpilling
    May 11 '21 at 1:42
  • @bpilling It was probably originally a bug that got fixed. The force:data:tree:export and force:data:tree:import are meant to be paired commands, and since the import only supports 200 records per file, the output files should also be 200 records each.
    – sfdcfox
    May 11 '21 at 2:17
  • @sfdcfox : Do you know why I would get 2000 records with my export (see my comment on the question), and yet @bpilling would not? I received the WARNING about using --plan, but I still got 2000 records in my JSON.
    – Moonpie
    May 11 '21 at 2:20
  • @Moonpie Maybe a difference in versions? Like I said, I think it's a bug that was fixed recently.
    – sfdcfox
    May 11 '21 at 2:33
  • 1
    Okay - I found something interesting here. So I stripped my query back to SELECT Id FROM ArticleTypeName__kav WHERE publishstatus='online' This then returned the full set - 1609. SO then I gradually built up the SOQL to what it was meant to be. It then stopped working as expected when I included a Rich Text field. Normal text was fine. So it seems that the rich text fields impact the ability to export. This certainly didn't use to be the case - because it was the rich text field content that was the most important for my use case.
    – bpilling
    May 14 '21 at 10:20
0

So after trying to figure this out for some time, I came to the conclusion this functionality is broken in SFDX. In the end I wrote my own jsforce node.js app to do the download and chunking myself.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.