Step 3 - Buyoff of results
As outlined before, upon completion of a task, the customer is typically notified about the task completion. As soon as the task is waiting for approval, the customer is notified by a call to the URL as defined by the <callback_url> element of the task-upload procedure (see 3.2.1). The body of this request will be a XML document carrying the necessary information for the customer to execute the buyoff.
BuyOff Notification
The following payload is sent by the notification to inform the customer that there is some action required:
- XML
<?xml version="1.0"?>
<notification>
<link
href="https://sandbox.clickworker.com/de/api/marketplace/v2/customer/tasks/2405829/
notifications/19265" rel="self" type="application/xml"/>
<event>CUSTOMER_INPUT_REQUIRED</event>
<task>
<link
href="https://sandbox.clickworker.com/de/api/marketplace/v2/customer/tasks/240582
9" rel="self" type="application/xml"/>
</task>
<job>
<link
href="https://sandbox.clickworker.com/de/api/marketplace/v2/customer/jobs/5968545
" rel="self" type="application/xml"/>
</job>
</notification>
As shown above the <job> element includes a <link> with an URL to the job which expects the buyoff. The customer now needs to judge the result quality.
With the given URL an http-get request will show the results for review:
- XML
<?xml version="1.0"?>
<job_response>
<request_status>
<id>RgsvEX3l3o8dS8pAjkQ02eXb2Ew=</id>
<valid>true</valid>
<status_code>200</status_code>
<status_text>OK</status_text>
</request_status>
<job>
<link
href="https://sandbox.clickworker.com/api/marketplace/v2/customer/jobs/5968547"
rel="self" type="application/xml"/>
<id>5968547</id>
<task>
<link
href="https://sandbox.clickworker.com/api/marketplace/v2/customer/tasks/2405830"
rel="task" type="application/xml"/>
</task>
<items>
<code>link_to_hotel</code>
<content>http://www.myhotel.com</content>
</items>
<items>
<code>title_top</code>
<content>Title text created by clickworker</content>
</items>
<items>
<code>paragraph_bottom</code>
<content>Paragraph text created by clickworker</content>
</items>
</job>
</job_response>
The various <items> elements within the
The items with <itemcode> ‘title_top’ and ‘paragraph_bottom’ are the output elements created by the _clickworker. These texts are the ones which need to be judged to approve or to reject these results.
Approval / Rejection
Approval
Assumed that the quality of the output elements meet the customer expectations, the job should be approved.
This job approval is executed by the posting of a XML document using the http-PUT method. The XML- document sent as payload of that http-PUT request is simple:
- XML
<job>
<input>
<items>
<code>accepted</code>
<content>1</content>
</items>
</input>
</job>
If you have indicated to get notified upon task completion, the notification will be sent to you as soon as you have approved the results (TASK_COMPLETED event). The payload of this notification is as follows. Therefore you do not need to pool for results.
- XML
<?xml version="1.0"?>
<notification>
<link type="application/xml" rel="self" href="
https://sandbox.clickworker.com/de/api/marketplace/v2/customer/tasks/2405829/notifi
cations/19266"/>
<event>TASK_COMPLETED</event>
<task>
<link type="application/xml" rel="self" href="
https://sandbox.clickworker.com/de/api/marketplace/v2/customer/tasks/2405829"/>
</task>
<customer_output>
<title_top>Title text created by clickworker</title_top>
<paragraph_bottom>Paragraph text created by clickworker</paragraph_bottom>
</customer_output>
</notification>
Within the <customer_output> element the output elements are embedded.
Rejection
Assumed that the result is not satisfying and does not comply with the briefing you may reject the job results. Rejecting a job is done by posting a XML document (http-PUT method). In comparison to the approval, the XML-Document needs some more elements to let the clickworker know why the job result has been rejected and allow for mitigation. Additionally as a requirement from the chosen product (within this chapter the Product “TextCreate” was chosen - see 3.1.2) individual aspects of the created text need to be judged:
- XML
<job>
<input>
<!-- Text is rejected: accepted=0 -->
<items>
<code>accepted</code>
<content> 0 </content>
</items>
<!--Judge grammar: 4=Perfect, 3=Very good, 2 = Ok, 0 = poor -->
<items>
<code>grade_gr</code>
<content>2</content>
</items>
<!--Comment your judge on grammar. This is obliged if you judged it with a value <
3 -->
<items>
<code>grade_gr_comment</code>
<content>Here is my grade_gr comment</content>
</items>
<!--Judge spelling and punctuation: 4=Perfect, 3=Very good, 2 = Ok, 0 = poor -->
<items>
<code>grade_sppu</code>
<content>2</content>
</items>
<!--Comment your judge on spelling and punctuation. This is obliged if you judged
it with a value < 3 -->
<items>
<code>grade_sppu_comment</code>
<content>Here is my grade_sppu comment</content>
</items>
<!--Judge sentence structure: 4=Perfect, 3=Very good, 2 = Ok, 0 = poor -->
<items>
<code>grade_sest</code>
<content>2</content>
</items>
<!--Comment your judge on sentence structure. This is obliged if you judged it
with a value < 3 -->
<items>
<code>grade_sest_comment</code>
<content>Here is my grade_sest comment</content>
</items>
<!--Judge the style: 4=Perfect, 3=Very good, 2 = Ok, 0 = poor -->
<items>
<code>grade_st</code>
<content>2</content>
</items>
<!--Comment your judge on style. This is obliged if you judged it with a value < 3
-->
<items>
<code>grade_st_comment</code>
<content>Here is my grade_st comment</content>
</items>
<!--Judge comprehension and clarity: 4=Perfect, 3=Very good, 2 = Ok, 0 = poor -->
<items>
<code>grade_cocl</code>
<content>2</content>
</items>
<!--Comment your judge on comprehension and clarity. This is obliged if you judged
it with a value < 3 -->
<items>
<code>grade_cocl_comment</code>
<content>Here is my grade_cocl comment</content>
</items>
<!--Judge the adherence to the working instructions: 4=Perfect, 3=Very good, 2 =
Ok, 0 = poor -->
<items>
<code>grade_foin</code>
<content>2</content>
</items>
<!--Comment your judge on adherence to the working instructions. This is obliged
if you judged it with a value < 3 -->
<items>
<code>grade_foin_comment</code>
<content>Here is my grade_foin comment</content>
</items>
<!--Judge selection of information: 4=Perfect, 3=Very good, 2 = Ok, 0 = poor -->
<items>
<code>grade_sein</code>
<content>2</content>
</items>
<!--Comment your judge on the selection of information. This is obliged if you
judged it with a value < 3 -->
<items>
<code>grade_sein_comment</code>
<content>Here is my grade_sein comment</content>
</items>
<!--Comment for the author. This is obliged.-->
<items>
<code>comment</code>
<content>Text ist in Ordnung.</content>
</items>
</input>
</job>
The following table lists the aspects for the judgement of the ‘TextCreate’ and ‘TextCreateWithKeywords’ Product. The column ‘item’ lists the <item_code> of the aspect. Column ‘meaning’ explains what the aspect means. Finally the third column ‘condition’ lists the circumstances under which the corresponding aspect must be included in the XML-Document executing the buyoff:
| item | meaning | condition |
| accepted | Is the result accepted? | Always. Either 0 (rejected) or 1 (accepted) |
| grade_gr | Judge the grammar. | If accepted=0. 4=Perfect, 3=Very good, 2 = Ok, 0 = poor |
| grade_gr_comment | Comment your Judge on grammar. | If grade_gr < 3. |
| grade_sppu | Judge spelling and punctuation. | If accepted=0. 4=Perfect, 3=Very good, 2 = Ok, 0 = poor |
| grade_sppu_comment | Comment your judge on spelling and punctuation. | If grade_sppu < 3. |
| grade_sest | Judge sentence structure. | If accepted=0. 4=Perfect, 3=Very good, 2 = Ok, 0 = poor |
| grade_sest_comment | Comment your judge on sentence | |
| structure. | If grade_sest < 3. | |
| grade_st | Judge the style. | If accepted=0. 4=Perfect, 3=Very good, 2 = Ok, 0 = poor |
| grade_st_comment | Comment your judge on style. | If grade_st < 3. |
| grade_cocl | Judge comprehension and clarity. | If accepted=0. 4=Perfect, 3=Very good, 2 = Ok, 0 = poor |
| grade_cocl_comment | Comment your judge on comprehension and clarity. | If grade_cocl < 3. |
| grade_foin | Judge the adherence to the working instructions | If accepted=0. 4=Perfect, 3=Very |
| good, 2 = Ok, 0 = poor | ||
| grade_foin_comment | Comment your judge on the adherence to the working instructions. | If grade_foin < 3. |
| grade_sein}Judge selection of information. | If accepted=0. 4=Perfect, 3=Very good, 2 = Ok, 0 = poor | |
| grade_sein_comment | Comment your judge on selection of information. | If grade_sein < 3. |
| comment | Comment your reason to reject the results. | If accepted=0. |
Table 1 Buyoff items
After the customer has executed a rejection, the clickworker will once get the chance to correct the results according to the customer’s comments. After the rework the customer is getting informed as before (see 3.3.1). The whole buyoff process restarts until the customer is doing the approval (see 3.3.2.1).
If a text is declined twice, the task will be canceled and ‘cloned’; the task will be handled by another clickworker. In this scenario the results ill be created for another task-id. As the customer only knows the ‘original’ task id (see 3.2.3), the parameter ‘ show_cloned_task=true ’ should always be part of the above-mentioned request URL. With this arameter the customer is always able to get the esults referencing the ‘original’ id.