logo   AtomServer, Batching Details

Chris Berry, Bryon Jacob. Updated   08/15/08

This document describes some specific details about batch processing in AtomServer. Batch processing gives you the ability to execute multiple operations in one request, rather than having to submit each operation individually.

For a further, detailed description of the actual protocol, either

Contents


General Information

Using a AtomServer batch feed, you can collect multiple insert, update, and delete operations, and then submit and execute them all at once. Unlike GData, we do not allow you to batch queries as well. Currently, we do not see a reasonable use case for batching queries, since feeds essentially provide this functionality already. 

For example, the following feed includes three operations (a delete and two updates -- one an insert and one an update)

<feed xmlns="http://www.w3.org/2005/Atom">
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <asbatch:operation type="update"/>
         <link href="/myserver/v1/widgets/acme/92347.en.xml/0" rel="edit"/>
         .... what to update ....
     </entry>
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <asbatch:operation type="delete"/>
         <link href="/myserver/v1/widgets/acme/92345.en.xml/1" rel="edit"/>
     </entry>
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <asbatch:operation type="update"/>
         <link href="/myserver/v1/widgets/acme/92348.en.xml" rel="edit"/>
         .... what to insert ....
     </entry>
</feed>

The service will perform as many of the requested changes as possible and return status information that you can use to evaluate the success or failure of each operation.

The service attempts to execute each of the operations within a batch, even if some of the operations included in the batch do not succeed. Note that this means that, internal to AtomServer, the operations are batched to the database. Thus, batch processing in AtomServer can very much improve overall performance, and is strongly recommended if you need to publish many entries at once.

There are several points to notice when looking at the example above;

Another important, but subtle point is that AtomServer allows "lazy inserts". This is because AtomServer, in keeping with REST principles, allows you to PUT an entry for either insert or update to the same URL, and AtomServer determines on-the-fly which operation is required. So, as expected, this behavior is reflected in the batch processing API, and thus, <asbatch:operation type="update"/> may be used for either an insert or an update, provided you supply an Entry URL in the "edit link" (e.g. <link href="/myserver/v1/widgets/acme/92348.en.xml/2" rel="edit"/>).

AtomServer also allows for typical RESTful POST operations, which are, by definition, insert operations. Again, you provide an "edit link", but, in this case, you will provide a "Feed URL. (e.g <link href="/myserver/v1/widgets/acme" rel="edit"/>).

Batch Entries are returned in the same order they were provided. And further Response details are supplied in the Entry, including  HTTP status code (200, 201, 400, 409, etc.) and what operation actually occured (insert, update, delete, or error)

Submitting a batch request

A batch request should be sent as an HTTP PUT to a AtomServer batch URL, which is simply the Feed URL with "/$batch" appended to it. So, for example, if you wanted to send a set of batch operations to the "widgets/acme" feed, then you would PUT to the following URL;

PUT /myserver/v1/widgets/$batch



Writing a batch operations feed

A batch operations feed contains a list of entries to insert, update, or delete. Each operation is defined by a <asbatch:operation type="insert|update|delete"/> element. How these these operations work directly reflects how AtomServer works for PUT, POST, and DELETE of individual Entries.

The operation element may be a direct child of a <feed> element, a direct child of any of the Entries in the Feed, or both. When included in an Entry, it specifies the operation to execute for that specific Entry. When included in the Feed, this element specifies the default operation to execute on any Entries that do not have a <asbatch:operation/> element.

When neither the Entry nor the Feed specifies an operation, the default operation is update.

Applications may not apply multiple operations to the same entry in a single batch feed. The results would be indeterminate if you specified multiple operations for the same entry, therefore, an Exception is thrown and you will recieve a 400 HTTP response code in the <asbatch:status/> element.

Operations are NOT necessarily performed in the order listed in the feed - operations are partitioned into two groups - updates and deletes - and each group is executed in a single batch operation to the database.  For this reason, you may not submit multiple operations on the same entry in the same batch, and doing so will produce an error for all duplicates. The first Entry in a duplicate set will be processed, and all others will result in errors.

It is important to note that AtomServer will attempt to process all Entries in the batch, and will return all Entries in the same order it received them. If any errors occured, those Entries will be returned along with all others, and will simply be flagged as errors. The status element will indicate the error that occured and a short description of the reason (e.g. <asbatch:status code="400" reason="Bad Request::....." /> )

By default, the number of entries in the XML batch operations feed that you send to the server may not exceed 15 for "full entries" and 100 for "link entries" (look here for definitions of these terms).  This restriction allows AtomServer to protect itself from excessive feeds. Note, the numbers shown here are the default values, and a AtomServer Workspace can be configured with alternate values.

Insert operations

An insert operation is denoted as follows:

<asbatch:operation type="insert">

In AtomServer, an insert operation is equivalent to either PUTing or POSTing a new Entry. Which of these occurs depends upon the "edit link" that is provided. If an Entry URL is provided within the "edit link"; a PUT equivalent occurs. And if a Feed URL is provided within the "edit link"; a POST equivalent occurs. This exactly matches the behavior of AtomServer in general. 

When the insert operation succeeds, the entire Entry content is returned, with an updated document id field. And you will receive a <asbatch:status code="201"/> element. 

Here is an example of a successful POST insert Request:

 <feed xmlns="http://www.w3.org/2005/Atom" xmlns:asbatch="http://atomserver.org/namespaces/1.0/batch">
    <entry xmlns:ha="http://atomserver.org/namespaces/1.0/">
         <content type="application/xml">
            <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92348" inNetwork="false">
               <contact>
                 <contactId>1638</contactId>
                 <displayName>Chris Berry</displayName>
                 <hasEmail>true</hasEmail>
               </contact>
             </property>
         </content>
         <asbatch:operation type="insert"/>
         <link href="/myserver/v1/widgets/acme" rel="edit"/>
     </entry>
</feed>

Here is an example of a Response to a successful insert request:

<?xml version='1.0' encoding='UTF-8'?>
<feed xmlns="http://www.w3.org/2005/Atom"
      xmlns:asbatch="http://atomserver.org/namespaces/1.0/batch">
    <asbatch:results inserts="1" updates="0" deletes="0" errors="0" />   
    <entry xmlns:as="http://atomserver.org/namespaces/1.0/">
         <id>/myserver/v1/widgets/acme/a371412926e0439983eda36651049dfa.en.xml</id>
         <updated>2007-11-15T17:09:19.000Z</updated>
         <published>2007-11-15T17:09:19.000Z</published>
         <title type="text">Entry: acme 92348.en</title>
         <author><name>AtomServer Atom Service</name></author>
         <link href="/myserver/v1/widgets/acme/a371412926e0439983eda36651049dfa.en.xml" rel="self"/>
         <link href="/myserver/v1/widgets/acme/a371412926e0439983eda36651049dfa.en.xml/1" rel="edit"/>
         <content type="application/xml">
            <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="a371412926e0439983eda36651049dfa" inNetwork="false">
               <contact>
                 <contactId>1638</contactId>
                 <displayName>Chris Berry</displayName>
                 <hasEmail>true</hasEmail>
               </contact>
             </property>
          </content>
         <category/>
         <asbatch:operation type="insert"/>
         <asbatch:status code="201" reason="OK"/>
     </entry>
</feed>

Note that the <asbatch:operation> is returned with type="insert" when an insert occurs.

Update operations

An update operation is denoted as follows:

<asbatch:operation type="update">

In AtomServer, a PUT operation can result in either an insert and an update operation, and not surprisingly AtomServer allows this same behavior in batch operations.

When the update operation succeeds, the entire entry content is returned, with an updated document id field. If an insert occurred, you will receive a <asbatch:status code="201"/> element. And if an update occurred, you will receive a <asbatch:status code="200"/> element.

Here is an example of a successful insert request:

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:asbatch="http://atomserver.org/namespaces/1.0/batch">
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <id>/myserver/v1/widgets/acme/92348.en.xml</id>
         <content type="application/xml">
            <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92348" inNetwork="false">
               <contact>
                 <contactId>1638</contactId>
                 <displayName>Chris Berry</displayName>
                 <hasEmail>true</hasEmail>
               </contact>
             </property>
         </content>
         <asbatch:operation type="update"/>
         <link href="/myserver/v1/widgets/acme/92348.en.xml/0" rel="edit"/>
     </entry>
</feed>

Here is an example of a response to a successful update request:

<feed xmlns="http://www.w3.org/2005/Atom"
      xmlns:asbatch="http://atomserver.org/namespaces/1.0/batch">
     <asbatch:results inserts="1" updates="0" deletes="0" errors="0" />   
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <id>/myserver/v1/widgets/acme/92348.en.xml</id>
         <updated>2007-11-15T17:09:19.000Z</updated>
         <published>2007-11-15T17:09:19.000Z</published>
         <title type="text">Entry: acme 92348.en</title>
         <author>
             <name>AtomServer Atom Service</name>
         </author>
         <link href="/myserver/v1/widgets/acme/92348.en.xml" rel="self"/>
         <link href="/myserver/v1/widgets/acme/92348.en.xml/1" rel="edit"/>
         <content type="application/xml">
            <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92348" inNetwork="false">
               <contact>
                 <contactId>1638</contactId>
                 <displayName>Chris Berry</displayName>
                 <hasEmail>true</hasEmail>
               </contact>
             </property>
          </content>
         <category/>
         <asbatch:operation type="insert"/>
         <asbatch:status code="201" reason="OK"/>
     </entry>
</feed>

Note that the <asbatch:operation> is returned with type="insert" when an insert occurs, and type="update" when an update occurs

Delete operations

<asbatch:operation type="delete">

A delete operation is equivalent to executing a DELETE on the URL referenced by by the entry's <next> element. For a delete operation, you only need to send a valid <link ... rel="next/> to delete the entry. Any other information you provide in elements that aren't in the asbatch: namespace will be ignored. When the operation succeeds, a <batch:status code="200"/> element will be returned.

Here is an example of a delete request:

<feed xmlns="http://www.w3.org/2005/Atom" xmlns:asbatch="http://atomserver.org/namespaces/1.0/batch">
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <asbatch:operation type="delete"/>
         <link href="/myserver/v1/widgets/acme/92345.en.xml/1" rel="edit"/>
     </entry>
</feed>

Here is an example of a successful response:

<feed xmlns="http://www.w3.org/2005/Atom"
      xmlns:asbatch="http://atomserver.org/namespaces/1.0/batch">
     <asbatch:results inserts="0" updates="0" deletes="1" errors="0" />   
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <id>/myserver/v1/widgets/acme/92345.en.xml</id>
         <updated>2007-11-15T17:09:19.000Z</updated>
         <published>2007-11-15T17:09:19.000Z</published>
         <title type="text">Entry: acme 92345.en</title>
         <author>
             <name>AtomServer Atom Service</name>
         </author>
         <link href="/myserver/v1/widgets/acme/92345.en.xml" rel="self"/>
         <link href="/myserver/v1/widgets/acme/92345.en.xml/2" rel="edit"/>
         <content type="application/xml">
             <deletion xmlns="http://schemas.atomserver.org/myserver/v1/rev0" collection="acme" id="92345" locale="en"
                       workspace="widgets">
               <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92345" inNetwork="false">
                  <contact>
                    <contactId>1638</contactId>
                    <displayName>Chris Berry</displayName>
                    <hasEmail>true</hasEmail>
                  </contact>
               </property>
            </deletion>
         </content>
         </content>
         <category/>
         <asbatch:operation type="delete"/>
         <asbatch:status code="200" reason="OK"/>
     </entry>


Handling status codes

Status codes are expressed by the following element:

<asbatch:status code="200|201|404|500|..." reason="reason" [content-type="type"]/>

Each entry in the response feed contains one <asbatch:status> element. This element describes what happened while executing the operation. It mimics the HTTP response that would have been sent if the operation had been sent individually, rather than as part of a batch feed.

You need to check the <asbatch:status> element of each entry in the response to find out whether the associated operation was successfully processed. The code="n" attribute contains a AtomServer status code.

Status descriptions

The reason="reason" attribute of the <asbatch:status> element contains a more verbose explanation of the operation's status.

Content type

The content-type="type" attribute of the <asbatch:status> element contains the MIME type of the data contained in the <batch:status> element. This corresponds to the Content-Type header of an HTTP status response. This attribute is optional.

When the content type is set, the body of the <asbatch:status> element describes what went wrong while processing the entry.


Example batch operations and status feed

Here is a batch operations feed that could be sent to the server. This feed requests that the server delete two entries and modify two new entries.


PUT   http://myserver.com/myserver/v1/widgets/acme/$batch
<feed xmlns="http://www.w3.org/2005/Atom">
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <content type="application/xml">
            <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92347" inNetwork="false">
               <contact>
                 <contactId>1638</contactId>
                 <displayName>Chris Berry</displayName>
                 <hasEmail>true</hasEmail>
               </contact>
             </property>
         </content>
         <asbatch:operation type="update"/>
         <link href="/myserver/v1/widgets/acme/92347.en.xml/1" rel="edit"/>
     </entry>
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <asbatch:operation type="delete"/>
         <link href="/myserver/v1/widgets/acme/92345.en.xml/1" rel="edit"/>
     </entry>
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <asbatch:operation type="delete"/>
         <link href="/myserver/v1/widgets/acme/23456.en.xml/5" rel="edit"/>
     </entry>
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <content type="application/xml">
            <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92348" inNetwork="false">
               <contact>
                 <contactId>1533</contactId>
                 <displayName>Bryon Jacob</displayName>
               </contact>
             </property>
         </content>
         <asbatch:operation type="update"/>
         <link href="/myserver/v1/widgets/acme/92348.en.xml/0" rel="edit"/>
     </entry>
</feed>

Let's assume that the two "updates" worked, but one of the two deletions failed. In this case, the batch status feed might look like the following. Note that the Entries are returned in exactly the same order that they appear in the Batch Request.


<feed xmlns="http://www.w3.org/2005/Atom"
      xmlns:asbatch="http://atomserver.org/namespaces/1.0/batch">
     <asbatch:results inserts="0" updates="2" deletes="1" errors="1" />   
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <id>/myserver/v1/widgets/acme/92347.en.xml</id>
         <updated>2007-11-15T17:09:19.000Z</updated>
         <published>2007-11-15T17:09:19.000Z</published>
         <title type="text">Entry: acme 92347.en</title>
         <author><name>AtomServer APP Service</name></author>
         <link href="/myserver/v1/widgets/acme/92347.en.xml" rel="self"/>
         <link href="/myserver/v1/widgets/acme/92347.en.xml/2" rel="edit"/>
         <content type="application/xml">
            <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92347" inNetwork="false">
               <contact>
                 <contactId>1638</contactId>
                 <displayName>Chris Berry</displayName>
                 <hasEmail>true</hasEmail>
               </contact>
             </property>
        </content>
        <asbatch:operation type="update"/>
        <asbatch:status code="200" reason="OK"/>
     </entry>
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <id>/myserver/v1/widgets/acme/92345.en.xml</id>
         <updated>2007-11-15T17:09:19.000Z</updated>
         <published>2007-11-15T17:10:19.000Z</published>
         <title type="text">Entry: acme 92345.en</title>
         <author><name>AtomServer APP Service</name></author>
         <link href="/myserver/v1/widgets/acme/92345.en.xml" rel="self"/>
         <link href="/myserver/v1/widgets/acme/92345.en.xml/2" rel="edit"/>
         <content type="application/xml">
             <deletion xmlns="http://schemas.atomserver.org/myserver/v1/rev0" collection="acme" id="92345" locale="en"
                       workspace="widgets">
               <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92345" inNetwork="false">
                  <contact>
                    <contactId>1533</contactId>
                    <displayName>Alex Victoria</displayName>
                  </contact>
                </property>
             </deletion>
         </content>
         <asbatch:operation type="delete"/>
         <asbatch:status code="200" reason="OK"/>
     </entry> 
     <entry>
         <id>/atomserver/v1/widgets/acme/23456.en.xml</id>
         <title type="text"> Entry: acme 23456.en</title>
         <author><name>AtomServer APP Service</name></author>
         <link href="/atomserver/v1/widgets/acme/23456.en.xml" rel="self" />
         <link href="/atomserver/v1/widgets/acme/23456.en.xml/2" rel="edit" />
         <asbatch:operation type="delete" />
         <asbatch:status code="404" reason="Unknown Entry:: /atomserver/v1/widgets/acme/$batch&#xa;Reason:: Entry [widgets, acme, 23456, en] NOT FOUND" />
     </entry>
     <entry xmlns:ha="http://atomserver.org/namespaces/atomserver/1.0/">
         <id>/myserver/v1/widgets/acme/92348.en.xml</id>
         <updated>2007-11-15T17:09:19.000Z</updated>
         <published>2007-11-15T17:11:19.000Z</published>
         <title type="text">Entry: acme 92348.en</title>
         <author><name>AtomServer APP Service</name></author>
         <link href="/myserver/v1/widgets/acme/92348.en.xml" rel="self"/>
         <link href="/myserver/v1/widgets/acme/92348.en.xml/1" rel="edit"/>
         <content type="application/xml">
            <property xmlns="http://schemas.atomserver.org/widgets/v1/rev0" systemId="acme" id="92348" inNetwork="false">
               <contact>
                 <contactId>1533</contactId>
                 <displayName>Bryon Jacob</displayName>
               </contact>
             </property>
         </content>
         <category/>
         <asbatch:operation type="insert"/>
         <asbatch:status code="200" reason="OK"/>
     </entry>
</feed>


Additional resources

You may find the following third-party documents useful:

    * Overview of Atom from IBM
    * HTTP 1.1 method definitions; specification for GET, POST, PUT, and DELETE
    * HTTP 1.1 status code definitions
    * Atom Syndication Reference (from Atom-enabled)
    * Getting to know the Atom Publishing Protocol (from IBM)