Taking data off Salesforce - but still be able to see it

Question

We got to a point, where our data usage has over our limit. More than 65% of that data is old/disabled data. However, that data should still be available to all users of SF to view (it hold's historical data). The idea was to take/export/delete all historical data (periodically) and upload it to another server, that can host it, with a SF visual page (or other option) to see the data.

One option I had was to install a Webservice on my IIS, and read the information that way. But that will make me develop all from scratch.

I thought of using any online storage sites (Amazon/Google) or even PAAS: Parse/Azure ?. But pretty much lost here.

Any ideas on solution that already can serve me this?

The solution should be of type: Full Monty! - Have the ability to query, browse, paging data , etc.

Answer 1

You might find the hugely exciting External Objects technology (currently in Pilot) quite interesting. It will allow you to store the data off platform but utilise a special kind of Custom Object to view it.

Track and represent data from sources outside Salesforce by creating external objects. Each external object maps to a table in a data source outside Salesforce, such as SharePoint®, and allows you to set up access to its data from search and custom tabs in Salesforce. Each external object relies on the connection details stored in an external data source definition.

Managing External Data Sources

Define external data sources to manage connections with data sources outside of Salesforce, such as SharePoint®. An external data source definition contains connection and authentication information to access external data. Once an external data source is set up, its visible content is mapped to external object definitions to enable searching for external data in Salesforce.

Answer 2

Keeping it in the Salesforce family to some degree...

Hosting and DB Choices. You might also want to consider the Heroku cloud platform (owned by Salesforce) to host a web application to surface the data from your external database. Heroku also supports a number of DB plugins (including Amazon RDS) and its own Postgres database hosted solution. There is even the option to consider Database.com, which has cheaper storage and also does have the benefit of allowing you to have the same objects structure (as it essentially a cut down Salesforce org), which will make your archive process easier to implement.

Building the UI. From a UI perspective, and following on from the Heroku hosted theme, I'd recommend the Spring MVC + Hibernate frameworks, its very easy to build data centric web UI's using this framework but does require HTML and Java skills obviously, here is some more detail. Here is a demo of it in action with a Database.com db backend.

Accessing the UI from within Salesforce. You can then use Salesforce Canvas technology to surface the application directly and securely inside the Salesforce UI as a tab or chatter publisher action. The option exists here to also place your web application within a Visualforce page, which gives you even more integration options, so for example a page embedded in a layout of an Account or other object, that allows contextual querying of related archived data.

Answer 3

in order to get the true full monty -- reporting, searching etc. Your only option that I know of is to increase your data storage limit with salesforce.

There are tools, DbAmp, etc. that can synch Salesforce and SqlServer, but these aren't going to let you then delete data from within Salesforce and access it from your external database.

Answer 4

To achieve this I would suggest you to move and Delete the Archive or Historical Databases from your Salesforce Instance. Once this is done you can make use of your Salesforce Efficiently.

Moving forward this job can be easily done through ETL Tools which are capable of doing things efficiently.