How to Import Plone Content into a CastleCMS Site

Use CastleCMS's included export and import scripts

CastleCMS includes scripts that export Plone content and import into CastleCMS.

As explained in the CastleCMS README, the overall process is as follows:

  • On the server for the Plone site, run the export script. This generates a directory containing all of the Plone site's content, in JSON format.
  • Copy the export directory to the server for the CastleCMS site.
  • On the server for the CastleCMS site, run the import script.

Each step is described in further detail below.

System Requirements

To export content from a Plone site, you must have the following access on both the Plone server and the CastleCMS server:

  • an account on the server (we'll refer to the account as either ploneserveruser or castleserveruser below)
  • ssh or shell access, so you can login or ssh to the server
  • sudo access, so you can read and write the necessary files

Exporting Plone Content

You must copy the export script into the main Plone installation directory tree. 

Here we assume that Plone was installed to the /opt/Plone/zeoserver directory using the Plone unified installer (using the "zeo" install option, i.e. ./install.sh zeo), which sets file ownership to the plone_buildout user and creates two ZEO clients, "client1" and "client2". 

We want to have the export script run in the same directory as the ZEO clients, so we change to the installation's "bin" folder before fetching the export script from the CastleCMS GitHub repository.

ssh to the Plone server,  then use sudo to become root, e.g. sudo -s

ssh ploneserveruser@ploneserver
sudo -s
cd /opt/Plone/zeoserver/bin
sudo -u plone_buildout wget https://raw.githubusercontent.com/castlecms/castle.cms/master/castle/cms/_scripts/export-content.py

To run the export script, you must use one of the Plone ZEO clients. (On production servers that are in active use, you may want to avoid using any ZEO clients that serve your users. Serious production Plone deployments typically create additional ZEO clients that are reserved for system administration or debugging purposes, e.g. "client_reserved" or "clientdebug", which you can easily create by editing the buildout.cfg and re-running buildout). In this example, however, we use the "client1" ZEO client:

./client1 run export-content.py --site-id=mysiteid --dir=./export

where "mysiteid" is the ID of your Plone site (usually "Plone") and "./export" is the directory that the script will create, in which it will create a file for every exported content item.

Exporting a Subset of the Plone Site

If you don't need to export the entire Plone site, you can modify the last line of the script: 

run_export(catalog())

so that instead of running against the entire contents of the Plone site catalog it runs over a subset that you define by providing a catalog query: *[need to test this!]

run_export(catalog({'portal_type' : 'Document'})) 

You should see lots of info messages scroll by, and hopefully no error messages, as the script fills the export directory with JSON files, one for each exported content item. The export directory tree will match the site's internal folder structure.

You can run the same command as often as you wish; the script will overwrite the contents of the export directory. 

Copying the Exported Files to the CastleCMS Server

To copy the exported files to the CastleCMS server, you create an archive containing the exported files, then you copy the archive to the CastleCMS server.

You may have to use an intermediate transfer destination computer if there is no direct ssh connection between the Plone and CastleCMS servers.

You may also have to use your home directory on each server as an intermediate transfer location if your user on each server cannot read from or write to the Plone or CastleCMS installation directory.

Create the Archive

Once the export is done, you will want to create a tar gzip'd archive of the export directory. 

Assuming you're in the same /opt/Plone/zeoserver/bin directory as before:

tar cfz export.tgz export

Copy the Archive to the CastleCMS Server

You must copy the resulting archive to the CastleCMS server. How you do that will depend on your system, network, and security settings.

Copying the Archive Directly to the CastleCMS Server

If your network and firewall settings allow it, the easiest is to copy the archive directly from the Plone server to the CastleCMS server:

scp export.tgz castleserveruser@castleserver:/opt/Plone/zeoserver/bin/

Using Your Computer as an Intermediate Transfer Point

If your network or firewall settings do not allow you to copy the archive directly from the Plone server to the CastleCMS server, you can download the archive to your computer, then, from your computer, upload the archive to the CastleCMS server. 

Use the scp command, which uses your ssh credentials on both servers.

For example, on your computer, issue a command like this:

scp ploneserveruser@ploneserver:/opt/Plone/zeoserver/bin/export.tgz .
If Your Plone Server User Doesn't Have Sufficient Permissions

If your user on the Plone server doesn't have read access to the /opt/Plone/zeoserver/bin directory, you may first have to copy the archive to your home directory.  To do this, use the following command on the server:

cp export.tgz ~ploneserveruser

By moving the archive to your user's home directory, you will be able to download it from there to your computer. On your computer:

scp ploneserveruser@ploneserver:export.tgz .

Upload the Archive to the CastleCMS Server

Once the archive is on your work computer, upload the archive from your work computer to the CastleCMS server:

scp export.tgz castleserveruser@castleserver:/opt/Plone/zeoserver/bin/

If your CastleCMS server user doesn't have write access to the /opt/Plone/zeoserver/bin directory you may have to first copy the archive to your home directory, then use sudo to move it to the ultimate destination directory:

scp export.tgz castleserveruser@castleserver:
ssh castleserveruser@castleserver
sudo mv export.tgz /opt/Plone/zeoserver/bin/

Expand the Archive

Once the archive is in its destination directory, you must expand it:

cd /opt/Plone/zeoserver/bin/
tar xfz export.tgz

This will recreate the export directory, containing all the exported Plone content items, in JSON format.

Import the Content into CastleCMS

On the CastleCMS server, you will need to copy the import script into your new CastleCMS site's main buildout folder.

From your computer:

ssh castleserveruser@castleserver
sudo -s
cd /opt/Plone/zeoserver/bin
wget https://raw.githubusercontent.com/castlecms/castle.cms/master/castle/cms/_scripts/importjson.py

Then, run the import script:

./client1 run importjson.py --site-id=mysiteid --export-directory=./export

The site ID you provide must be for an existing CastleCMS; the import script will not create the site for you. You can create a new CastleCMS site by browsing to the server, usually at http://castleserver:8080 (though you may need to ssh tunnel) *[explain this]

You can run the import script as often as needed; it will delete any matching existing items and recreate them.

You will likely see an error about a conflicting ID for front-page. 

Importing into an Existing CastleCMS Site

If you have a CastleCMS site into which you'd like to import the Plone content but you want to be careful not to overwrite existing content, you should import the content into a new, otherwise empty, CastleCMS site, then copy and paste the imported content into the final destination CastleCMS site.

For example, if your existing CastleCMS site has the ID MyCastleCMSSite

ssh castleserveruser@castleserver -L8080:localhost:8080

Browse to http://localhost:8080 and create a new CastleCMS site having the ID WorkingImport.

Then run the import script like this:

cd /opt/Plone/zeoserver/bin
./client1 run importjson.py --site-id=WorkingImport --export-directory=./export

Once the import is complete, browse to http://localhost:8080/MyCastleCMSSite and then, in another browser tab or window, browse to http://localhost:8080/WorkingImport

In the WorkingImport browser tab, use the Contents button and navigate to the content items you want to copy. After you've selected the content items, use the Copy button.

In the MyCastleCMSSite browser tab, use the Contents button and navigate to the folder where you want to paste the items. (If you're already in that destination folder, you should click the Reload button so that the content items you copied are available for pasting). Use the Paste button to paste the items.

Repeat this process until you've copied and pasted all the content items you wanted to move.