NAV
ControlShift Labs

Introduction

Welcome to the ControlShift Labs Developer documentation. Our APIs are designed to allow software engineers to create rich integrations between ControlShift content and third-party services and websites.

We offer several APIs to customers to cover different integration use cases.

JSON API Endpoints

The JSONP API is a simple way to embed ControlShift petition content in external sites. It’s intended for use by a front-end developer embed content on web pages outside of the platform. For example, a developer could:

All of the endpoints can be consumed as JSONP instead of JSON by adding callback or variable parameters to the URLs.

The URL slugs through the API are the same as those that are used through the web to represent specific petitions or categories. Many front-end libraries including jQuery make it easy to consume JSONP endpoints. Our examples below use jQuery to consume these resources.

Get a single petition

$(document).ready(function(){
  var petitionSlug = 'turn-back-human-trafficking-faith-leaders';
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/petitions/'+petitionSlug+'.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would display your returned petition data in the console. The JSON response data would be structured like this:

{
  "id": 107370,
  "title": "Pledge to TURN BACK human trafficking Now - Faith or Community Leaders",
  "who": "President John Dramani Mahama",
  "why": "Today there are over 192,000 people trapped in modern slavery in Ghana [1]. Denied of their rights to education and often lacking parental care or support, over 1.8 million Ghanaian children are working rather than in school (21.8% of the population). Of these an estimated 1.2 million are forced to undertake hazardous child labour including mining, fishing and domestic slavery [3].\r\n\r\nThe experience of Kwesi is a typical example. Following the death of his father, Kwesi’s mother borrowed money to pay for the funeral. Faced with a debt she was unable to pay her creditor threatened her with jail unless she gave up her son as a labourer.  Kwesi was trafficked to Lake Volta to work as a “fisher boy”. and for the next three years, Kwesi worked for an average of 18 hours per day with little rest, no pay and only enough food to give him the energy for more work. By the time we found Kwesi and rescued him, he had suffered extensive emotional and physical abuse and partially lost sight in his left eye. \r\n\r\nOver 21,000 children like Kwesi, many of them trafficked, are trapped in hazardous labour on Lake Volta in Ghana[2]. We are working hard to bring them home, but want to ensure no one is trafficked in the first place. Please help fulfil the wish made by Kwesi when he gained his freedom: “I want the world to know that no child should be sent into slavery”.\r\n\r\n[1]. http://www.globalslaveryindex.org/ 2014\r\n[2].International Labour Organisation/International Programme on Elimination of Child Labour (ILO/IPEC) Analytical Study on Child Labour In Lake Volta Fishing in Ghana\r\n[3].Ghana Living Standard Survey 6 (GLSS 6): Child Labour Report by Ghana Statistical Services, 2014",
  "what": "I know that globally millions of people are trafficked for forced labour and sexual exploitation and that Ghana is a source, transit and destination country for this social evil. The concerns raised by Parliament recently, about men, women, and children who are trafficked and subjected to forced labor and sexual exploitation human points to the need for everyone to take action.\r\n\r\nI hereby pledge to TURN BACK human trafficking. \r\n•\tI will be vigilant of suspicious movement of children and vulnerable people, victims of bonded labour and of children working when they should be in school \r\n•\tI promise to always report my concerns to the appropriate authorities as quickly as possible.\r\n\r\nAs a faith or community leader, I promise to warn my congregation of the dangers of child trafficking, and support them in reporting any suspicions they have.",
  "created_at": "2015-06-01T15:37:47.621-04:00",
  "updated_at": "2015-10-14T08:45:52.396-04:00",
  "slug": "turn-back-human-trafficking-faith-leaders",
  "delivery_details": null,
  "administered_at": "2015-07-29T09:32:08.258-04:00",
  "source": "",
  "location_id": null,
  "alias": null,
  "bsd_constituent_group_id": null,
  "categories": [{
    "name": "Civil Rights",
    "slug": "civil-rights-1"
  }],
  "goal": 100,
  "effort": null,
  "group": "turn-back-human-trafficking-now",
  "resized_image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/hero/Untitled.png?1433187465",
  "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/original/Untitled.png?1433187465",
  "creator_name": "Kristyn Arrighi",
  "signature_count": 6
}

This retrieves a single petition object.

HTTP Request

GET https://demo.controlshiftlabs.com/petitions/<slug>.json

Query Parameters

Parameter Default Description
slug null string - required - The petition’s unique identification slug. If none is provided, you will get a 404 error. Note: submitted as a part of the endpoint path, not as a separate URL parameter

Working Example

View and edit a working example on codepen.io:

$(document).ready(function(){
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/petitions/featured.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would display your returned featured petitions data in the console. The JSON would be structured like this:

{
  "meta": {
    "total_pages": 3,
    "total_entries": 25,
    "per_page": 10
  },
  "data": [
  {
    "slug": "release-pro-democracy-youth-activists-in-azerbaijan",
    "title": "Release pro-democracy youth activists in Azerbaijan",
    "url": "http://demo.controlshiftlabs.com/petitions/release-pro-democracy-youth-activists-in-azerbaijan",
    "admin_status": "awesome",
    "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/hero/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011",
    "additional_image_sizes_url": [{
      "style": "form",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/form/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011"
    }, {
      "style": "large",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/large/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011"
    }, {
      "style": "open_graph",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/open_graph/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011"
    }, {
      "style": "original",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/original/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011"
    }],
    "who": "President Ilham Aliyev of Azerbaijan",
    "what": "Immediately release the imprisoned youth activists from the NIDA civic movement.\r\n\r\nShahin Novruzlu, Rashadat Akundov, Mammad Azizov, Bakhtiyar Guliyev, Zaur Gurbanli, Rashad Hasanov, Uzeyir Mammadli and Ilkin Rustamzade were arrested on criminal charges of possessing explosives and intent of public disorder. Human rights organizations believe these charges have been fabricated. The actual reason for their arrest was their criticism of the Azerbaijani authorities and their online activism, calling for peaceful demonstrations.\r\n\r\nShahin Novruzlu, who was only 17 at the time of his arrest, has lost his front teeth as a result of torture in detention. Mammad Azizov and Bakhtiyar Guliyev have complained of torture too.\r\n\r\nFrom 14 May 2014, Azerbaijan is chairing the Committee of Ministers of the Council of Europe. This organization is responsible for promoting and protecting human rights. I am deeply concerned that there are civil society activists in Azerbaijan tortured and imprisoned solely for expressing their views and organizing peaceful protests.\r\n\r\nAzerbaijan must live up to the high expectations of the chair of the Committee of Ministers of the Council of Europe. The NIDA activists must be immediately released.",
    "goal": 200,
    "signature_count": 119,
    "creator_name": "Kristyn Arrighi",
    "created_at": "2014-07-24T20:07:14.607-04:00",
    "updated_at": "2016-02-26T14:30:57.330-05:00",
    "why": "Shahin Novruzlu and fellow youth activists from the pro-democracy movement NIDA campaigned for democracy and against human rights abuses and widespread corruption in Azerbaijan. In February 2013 th..."
  }, {
    "slug": "give-us-more-icecream",
    "title": "Give us more icecream",
    "url": "http://demo.controlshiftlabs.com/petitions/give-us-more-icecream",
    "admin_status": "awesome",
    "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/hero/10982478_634745530004482_279346285079955699_n.jpg?1455250695",
    "additional_image_sizes_url": [{
      "style": "form",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/form/10982478_634745530004482_279346285079955699_n.jpg?1455250695"
    }, {
      "style": "large",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/large/10982478_634745530004482_279346285079955699_n.jpg?1455250695"
    }, {
      "style": "open_graph",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/open_graph/10982478_634745530004482_279346285079955699_n.jpg?1455250695"
    }, {
      "style": "original",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/original/10982478_634745530004482_279346285079955699_n.jpg?1455250695"
    }],
    "who": "Cook",
    "what": "Cook and provide more icecream",
    "goal": 100,
    "signature_count": 6,
    "creator_name": "Tudor Bradatan",
    "created_at": "2016-02-11T23:17:42.782-05:00",
    "updated_at": "2016-02-26T14:20:46.987-05:00",
    "why": "Because we like and need it",
    "location": {
      "query": "Cluj",
      "latitude": "46.7712101",
      "longitude": "23.6236353",
      "street": "",
      "postal_code": "",
      "country": "RO",
      "region": "CJ",
      "street_number": "",
      "venue": "",
      "created_at": "2016-02-11T23:17:42.769-05:00"
    }
  }, {
    "slug": "turn-back-human-trafficking-faith-leaders",
    "title": "Pledge to TURN BACK human trafficking Now - Faith or Community Leaders",
    "url": "http://demo.controlshiftlabs.com/petitions/turn-back-human-trafficking-faith-leaders",
    "admin_status": "awesome",
    "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/hero/Untitled.png?1433187465",
    "additional_image_sizes_url": [{
      "style": "form",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/form/Untitled.png?1433187465"
    }, {
      "style": "large",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/large/Untitled.png?1433187465"
    }, {
      "style": "open_graph",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/open_graph/Untitled.png?1433187465"
    }, {
      "style": "original",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/original/Untitled.png?1433187465"
    }],
    "who": "President John Dramani Mahama",
    "what": "I know that globally millions of people are trafficked for forced labour and sexual exploitation and that Ghana is a source, transit and destination country for this social evil. The concerns raised by Parliament recently, about men, women, and children who are trafficked and subjected to forced labor and sexual exploitation human points to the need for everyone to take action.\r\n\r\nI hereby pledge to TURN BACK human trafficking. \r\n•\tI will be vigilant of suspicious movement of children and vulnerable people, victims of bonded labour and of children working when they should be in school \r\n•\tI promise to always report my concerns to the appropriate authorities as quickly as possible.\r\n\r\nAs a faith or community leader, I promise to warn my congregation of the dangers of child trafficking, and support them in reporting any suspicions they have.",
    "goal": 100,
    "signature_count": 6,
    "creator_name": "Kristyn Arrighi",
    "created_at": "2015-06-01T15:37:47.621-04:00",
    "updated_at": "2015-10-14T08:45:52.396-04:00",
    "why": "Today there are over 192,000 people trapped in modern slavery in Ghana [1]. Denied of their rights to education and often lacking parental care or support, over 1.8 million Ghanaian children are wo..."
  },
  // ...
],
"links": {
    "self": "https://demo.controlshiftlabs.com/petitions/featured.json?page=1",
    "first": "https://demo.controlshiftlabs.com/petitions/featured.json",
    "prev": null,
    "next": "https://demo.controlshiftlabs.com/petitions/featured.json?page=2",
    "last": "https://demo.controlshiftlabs.com/petitions/featured.json?page=3"
  }

This retrieves a JSON object compliant with the JSON API specification, containing an array of featured petitions objects on its data property.

Important Note: The previous featured petitions API endpoint at https://demo.controlshiftlabs.com/featured.json (note the missing /petitions path) is being deprecated and will be removed in the near future. Please use this new endpoint for retrieving featured petitions from now on.

HTTP Request

GET https://demo.controlshiftlabs.com/petitions/featured.json

Query Parameters

Parameter Default Description
locale null string - optional - Locale filter for petitions

Working Example

View and edit a working example on codepen.io:

Get list of categories

$(document).ready(function(){
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/categories.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would display a list of categories in the console. The JSON would be structured like this:

[
  {
    "category_name": "Animal Rights",
    "category_count": 7,
    "slug": "animal-rights-2",
    "url": "https://demo.controlshiftlabs.com/categories/animal-rights-2.json",
    "signature_count": 8,
    "locales": {
      "es": "Derechos Animales"
    }
  }, {
    "category_name": "cats",
    "category_count": 31,
    "slug": "cats",
    "url": "https://demo.controlshiftlabs.com/categories/cats.json",
    "signature_count": 270,
    "locales": {
      "es": "gatos"
    }
  }, {
    "category_name": "Civil Rights",
    "category_count": 23,
    "slug": "civil-rights-1",
    "url": "https://demo.controlshiftlabs.com/categories/civil-rights-1.json",
    "signature_count": 165,
    "locales": {
      "es": "Derechos Civiles"
    }
  }, {
    "category_name": "Culture",
    "category_count": 3,
    "slug": "cultura",
    "url": "https://demo.controlshiftlabs.com/categories/cultura.json",
    "signature_count": 10,
    "locales": {
      "es": "Cultura"
    }
  }
]

This retrieves a JSON array of category objects.

HTTP Request

GET https://demo.controlshiftlabs.com/categories.json

Working Example

View and edit a working example on codepen.io:

List petitions in a category

$(document).ready(function(){
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/categories/mice.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would return petitions data from the category with the slug mice. The JSON would be structured like this:

{
  "current_page": 1,
  "total_pages": 1,
  "previous_page": null,
  "next_page": null,
  "name": "mice",
  "results": [{
    "slug": "apple-stop-your-comic-sans-snobbery",
    "title": "Apple: Stop your Comic Sans snobbery",
    "url": "http://demo.controlshiftlabs.com/petitions/apple-stop-your-comic-sans-snobbery",
    "admin_status": "good",
    "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/hero/helveticacomicsans.gif?1398764339",
    "additional_image_sizes_url": [{
      "style": "form",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/form/helveticacomicsans.gif?1398764339"
    }, {
      "style": "large",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/large/helveticacomicsans.gif?1398764339"
    }, {
      "style": "open_graph",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/open_graph/helveticacomicsans.gif?1398764339"
    }, {
      "style": "original",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/original/helveticacomicsans.gif?1398764339"
    }],
    "who": "Tim Cook, CEO Apple",
    "what": "Use Comic Sans for the default font on the iPhone 6.",
    "goal": 100,
    "signature_count": 7,
    "creator_name": "John Wood",
    "created_at": "2014-04-29T05:35:47.693-04:00",
    "updated_at": "2016-01-06T11:02:17.617-05:00",
    "why": "Apple occupy a key position in the hearts of typeface poseurs everywhere. If they were to make a bold step and switch to using the people's font, Comic Sans on the new iPhone, this would cause mass..."
  }, {
    "slug": "clean-up-the-himalayas",
    "title": "Clean up the Himalayas",
    "url": "http://demo.controlshiftlabs.com/petitions/clean-up-the-himalayas",
    "admin_status": "good",
    "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/hero/gorak_shep_thumb.jpg?1431734031",
    "additional_image_sizes_url": [{
      "style": "form",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/form/gorak_shep_thumb.jpg?1431734031"
    }, {
      "style": "large",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/large/gorak_shep_thumb.jpg?1431734031"
    }, {
      "style": "open_graph",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/open_graph/gorak_shep_thumb.jpg?1431734031"
    }, {
      "style": "original",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/original/gorak_shep_thumb.jpg?1431734031"
    }],
    "who": "Nepal Ghurung",
    "what": "Clean up the Khumbu region.",
    "goal": 100,
    "signature_count": 1,
    "creator_name": "Grant Cox",
    "created_at": "2013-06-03T17:13:41.399-04:00",
    "updated_at": "2015-07-17T15:11:48.665-04:00",
    "why": "It's quite dirty, and I don't like it.",
    "location": {
      "query": "Nepal",
      "latitude": "28.394857",
      "longitude": "84.124008",
      "street": "",
      "postal_code": "",
      "country": "NP",
      "region": "",
      "street_number": "",
      "venue": "Nepal",
      "created_at": "2013-06-03T17:13:41.057-04:00"
    }
  }]
}

This retrieves a paginated list of petitions in a category.

HTTP Request

GET https://demo.controlshiftlabs.com/categories/<category slug>.json

Query Parameters

Parameter Default Description
category slug null string - required - submitted as a part of the endpoint path, not as a separate URL parameter
page 1 integer - optional - The page number of results for the specified category. Minimum of 1.

Working Example

View and edit a working example on codepen.io:

List petitions in an effort

$(document).ready(function(){
  var effortSlug = 'forecast-the-facts';
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/efforts/'+effortSlug+'.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would return petitions data from the effort with the slug forecast-the-facts. The JSON would be structured like this:

{
  "title": "Forecast the Facts",
  "slug": "forecast-the-facts",
  "description": "It is imperative that weather reporters fulfill an important part of their job, and explain the ways that climate change has influenced the freakish \"Frankenstorm\" Sandy. Below is a list of the meteorologists in your area, with telephone numbers and e-mail addresses. Please call or email them a message in your own words, asking them to connect the dots between global warming pollution and disasters like Sandy. Take a look at the talking points below if you need any help.",
  "goal": 100,
  "signature_count": 11,
  "image_url": "https://d8s293fyljwh4.cloudfront.net/efforts/images/16/hero/forecast.png?1351625402",
  "petitions": [
    {
      "slug": "forecast-the-facts-of-frankenstorm-wmtv-madison-wi",
      "title": "Forecast the Facts of Frankenstorm: WMTV Madison, WI",
      "url": "http://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-wmtv-madison-wi",
      "admin_status": "good",
      "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/hero/forecast.png?1429199472",
      "additional_image_sizes_url": [{
        "style": "form",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/form/forecast.png?1429199472"
      }, {
        "style": "large",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/large/forecast.png?1429199472"
      }, {
        "style": "open_graph",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/open_graph/forecast.png?1429199472"
      }, {
        "style": "original",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/original/forecast.png?1429199472"
      }],
      "who": "WMTV Madison, WI",
      "what": "It is imperative that weather reporters fulfill an important part of their job, and explain the ways that climate change has influenced the freakish \"Frankenstorm\" Sandy. Below is a list of the meteorologists in your area, with telephone numbers and e-mail addresses. Please call or email them a message in your own words, asking them to connect the dots between global warming pollution and disasters like Sandy. Take a look at the talking points below if you need any help.",
      "goal": 100,
      "signature_count": 1,
      "creator_name": "Kristyn Arrighi",
      "created_at": "2015-04-16T11:51:12.700-04:00",
      "updated_at": "2015-08-03T20:23:06.034-04:00",
      "why": "We want TV meteorologists to understand that the public is depending on them for information about climate change, especially as it increasingly impacts our weather patterns. We know that meteorolo...",
      "location": {
        "query": "WMTV-TV (Madison), Forward Drive, Madison, WI, United States",
        "latitude": "43.05091",
        "longitude": "-89.486738",
        "street": "Forward Dr",
        "postal_code": "53711",
        "country": "US",
        "region": "WI",
        "street_number": "615",
        "venue": "WMTV-TV (Madison)",
        "created_at": "2015-04-16T11:51:11.222-04:00"
      }
    }, {
      "slug": "forecast-the-facts-of-frankenstorm-woak",
      "title": "Forecast the Facts of Frankenstorm: WOAK",
      "url": "http://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-woak",
      "admin_status": "good",
      "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/hero/forecast.png?1439846499",
      "additional_image_sizes_url": [{
        "style": "form",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/form/forecast.png?1439846499"
      }, {
        "style": "large",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/large/forecast.png?1439846499"
      }, {
        "style": "open_graph",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/open_graph/forecast.png?1439846499"
      }, {
        "style": "original",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/original/forecast.png?1439846499"
      }],
      "who": "WOAK",
      "what": "It is imperative that weather reporters fulfill an important part of their job, and explain the ways that climate change has influenced the freakish \"Frankenstorm\" Sandy. Below is a list of the meteorologists in your area, with telephone numbers and e-mail addresses. Please call or email them a message in your own words, asking them to connect the dots between global warming pollution and disasters like Sandy. Take a look at the talking points below if you need any help.",
      "goal": 100,
      "signature_count": 2,
      "creator_name": "Adam Klaus",
      "created_at": "2015-08-17T17:21:39.791-04:00",
      "updated_at": "2015-08-17T17:29:02.816-04:00",
      "why": "We want TV meteorologists to understand that the public is depending on them for information about climate change, especially as it increasingly impacts our weather patterns. We know that meteorolo...",
      "location": {
        "query": "Oklahoma City, OK, United States",
        "latitude": "35.4675602",
        "longitude": "-97.5164276",
        "street": "",
        "postal_code": "",
        "country": "US",
        "region": "OK",
        "street_number": "",
        "venue": "",
        "created_at": "2013-12-17T14:49:51.324-05:00"
      }
    }
  ]
}

This retrieves a list of petitions in an effort.

HTTP Request

GET https://demo.controlshiftlabs.com/efforts/<effort slug>.json

Parameter Default Description
effort slug null string - required - submitted as a part of the endpoint path, not as a separate URL parameter

Working Example

View and edit a working example on codepen.io:

Find near by petitions

$(document).ready(function(){
  var effortSlug = 'forecast-the-facts',
    latitude = 38.88233400000001,
    longitude = -77.17109140000002;
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/efforts/'+effortSlug+'/'+'near.json',
    dataType: 'jsonp',
    data: {
      'location[latitude]': latitude,
      'location[longitude]': longitude
    }
  })
  .done(function( data ) {
    console.log(data);
  });
});

The above code would return petitions data from the effort with the slug forecast-the-facts. The JSON would be structured like this:

{
  "closest_target": {
    "slug": "doug-kammerer-nbc4",
    "name": "Doug Kammerer - NBC4",
    "location": "Washington D.C., DC",
    "status": "target_petition_created",
    "petition": {
      "slug": "forecast-the-facts-of-frankenstorm-doug-kammerer-nbc4",
      "title": "Forecast the Facts of Frankenstorm: Doug Kammerer - NBC4",
      "url": "https://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-doug-kammerer-nbc4",
      "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/2139/hero/forecast.png?1351625677",
      "who": "Doug Kammerer - NBC4",
      "goal": 100,
      "signature_count": 1
    }
  },
  "other_targets": [
    {
      "slug": "linda-church-wpix",
      "name": "Linda Church - WPIX",
      "location": "New York, NY",
      "status": "target_petition_created",
      "petition": {
        "slug": "forecast-the-facts-of-frankenstorm-linda-church-wpix",
        "title": "Forecast the Facts of Frankenstorm: Linda Church - WPIX",
        "url": "https://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-linda-church-wpix",
        "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/2137/hero/forecast.png?1351625520",
        "who": "Linda Church - WPIX",
        "goal": 100,
        "signature_count": 1
      }
    },
    {
      "slug": "wnyc",
      "name": "WNYC ",
      "location": "Manhattan, New York, NY, United States",
      "status": "target_awaiting_petition",
      "create_petition_url": "https://demo.controlshiftlabs.com/efforts/forecast-the-facts/petitions/creating?target_id=5148"
    },
    {
      "slug": "whdtv-boston",
      "name": "WHDTV Boston",
      "location": "Boston, MA, United States",
      "status": "target_petition_created",
      "petition": {
        "slug": "forecast-the-facts-of-frankenstorm-whdtv-boston",
        "title": "Forecast the Facts of Frankenstorm: WHDTV Boston",
        "url": "https://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-whdtv-boston",
        "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28044/hero/forecast.png?1405015225",
        "who": "WHDTV Boston",
        "goal": 100,
        "signature_count": 1
      }
    },
    {
      "slug": "wmtv-madison-wi",
      "name": "WMTV Madison, WI",
      "location": "WMTV-TV (Madison), Forward Drive, Madison, WI, United States",
      "status": "target_petition_created",
      "petition": {
        "slug": "forecast-the-facts-of-frankenstorm-wmtv-madison-wi",
        "title": "Forecast the Facts of Frankenstorm: WMTV Madison, WI",
        "url": "https://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-wmtv-madison-wi",
        "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/hero/forecast.png?1429199472",
        "who": "WMTV Madison, WI",
        "goal": 100,
        "signature_count": 1
      }
    }
  ]
}

Search for closest petition in an effort

This JSON endpoint allows you to reproduce the “search for the nearest petition in an effort” interface. Use this endpoint to build a place for users to enter their location. Once they’ve searched for a location, we’ll return the nearest petition. This endpoint can most easily be used with a Google Maps integration to help your users locate their addresses or other points of interest. For an example, click on the working example below.

HTTP Request

GET https://demo.controlshiftlabs.com/efforts/<effort slug>/near.json

Query Parameters

This endpoints accepts a few nested attributes in a location object, as described below.

Parameter Default Description
effort slug null string - required - submitted as a part of the endpoint path, not as a separate URL parameter
location[latitude] null float - required - a float representing the queried latitude
location[longitude] null float - required - a float representing the queried longitude
location[country] null string - optional - for efforts configured to use the “by country” search strategy, you can pass in the target country as a two letter ISO 3166-1 code.
location[region] null string - optional - for efforts configured to use the “by state” search strategy, you can pass in the target state as a two letter state abbreviation.

Working Example

View and edit a working example on codepen.io:

Logged in user

$(document).ready(function(){
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/api/graph/me.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would user details for a signed in user. The returned JSON would be structured like this:

{
  "status":"authenticated",
  "me":{
    "first_name":"John",
    "last_name":"Smith",
    "email":"john@smith.com"
  },
  "petitions":[
    {
       "slug":"sample-campaign",
       "title":"Sample Campaign",
       "created_at":"2016-03-25T15:37:28.849-04:00",
       "updated_at":"2016-03-25T15:37:35.304-04:00"
    }
  ],
  "events":[
    {
       "slug":"sample-campaign",
       "title":"Sample Campaign",
       "created_at":"2016-03-25T15:37:28.849-04:00",
       "updated_at":"2016-03-25T15:37:35.304-04:00"
    }
  ],
  "chapters":[
    {
       "name":"Brooklyn bruisers",
       "slug":"brooklyn-bruisers",
       "created_at":"2015-05-14T15:53:03.901-04:00",
       "updated_at":"2016-03-25T15:40:44.661-04:00"
    }
  ]
}

Response for a visitor who is not logged in:

{
  "status":"not_authenticated",
  "message":"user is not signed in"
}

Response for a hostname that is not whitelisted:

{
  "status":"error",
  "errors":[
    {
      "field":"referer",
      "messages":[
        "host name not whitelisted"
      ]
    }
  ]
}

We provide a specialized jsonp endpoint to return information about the currently signed in user. Customers can use this endpoint to power features where information about the current user’s petitions, sign in status, and other information about their account is displayed on external sites. Some examples of the sorts of functionality that this could be used to provide include:

HTTP Request

GET http://demo.controlshiftlabs.com/api/graph/me.json

Working Example

View and edit a working example on codepen.io. Note: codepen.io is whitelisted for the demo account.

Whitelisting a Hostname

We authenticate requests to the current user endpoint by validating the hostname set in the HTTP referrer header against a whitelist. The whitelist is set of hostnames that scripts can be served from. If you request the endpoint from a script served from a non-whitelisted hostname you’ll get an error message. This is necessary to prevent cross site scripting attacks that would allow someone to display logged in user information on unauthorized sites.

You can add hostnames to the whitelist through a self-serve interface in the “Settings” area of your instance. Settings > Add Hostname > enter a Hostname, and click save.

The hostname must be a hostname string like “localhost” “www.google.com” or the like. Wildcards or full HTTP URLs are not supported.

When you access the graph API endpoint from a page served from a whitelisted hostname the browser will automatically set a referrer header on those requests. We use this referrer header to validate the server the script was served from against the whitelist. You can verify that a whitelisted hostname is working properly with the curl command line tool:

curl --referer http://localhost/ http://demo.controlshiftlabs.com/api/graph/me.json

The above should return a JSON blob indicating that the user is not signed in if localhost is whitelisted. Otherwise you’ll get a not whitelisted error.

For authenticated users we currently return their first and last names, email addresses, and lists of petitions they’ve created, events they’ve created and local groups they are a member of.

Webhook Endpoints

Webhooks can be used by software engineers to integrate ControlShift with third-party systems. They allow engineers to build software that is triggered by events that take place within ControlShift. ControlShift would execute HTTP calls when certain event happen (e.g. a petition is launched; a category is changed).

To begin using these new webhooks, go to the admin homepage and click “Settings” under “Manage.” Then choose the “CRM Integration & Webhooks” tab and click “Configure Webhook Endpoints.” To begin sending data, you’ll need to add a “New Webhook Endpoint.” Each Webhook Endpoint URL receives a full firehose of all hooks that occur within ControlShift.

We recommend using a tool like RequestBin to help debug and develop your webhook integration. RequestBin allows engineers to see all of the webhooks generated by the application and sent to an endpoint.

If you need additional information about webhooks or how to use them, please send us a support email support@controlshiftlabs.com. We’d also love stories about how customers are using webhooks.

Retries

For webhook notifications successfully processed the response should have an HTTP 2XX status code. Responses returning a different status code will be marked as failed and automatically retried up to 9 times with an exponential backoff (e.g.: after 15s, 30s, 90s, etc.).

If after 9 retries the notification cannot be delivered, the webhook endpoint will be automatically disabled and will need to be re-enabled from the admin page to start receiving webhook notifications again.

Webhook types summary

You can configure ControlShift Labs Webhooks to return the following event types:

Type Description
blast_email.created A new blast email is ready for moderation
category.created A new category is created
category.updated A category is changed
category.deleted A category has been deleted and removed from the organisation
data.full_table_exported A full-table dump is ready for retrieval at S3
data.incremental_table_exported An incremental CSV update is ready for retrieval at S3
event.created A new event is published
event.updated An event is updated
attendee.created Member RSVP to event
attendee.updated Member updates RSVP to event
attendee.deleted Attendee permanently deleted. This is currently only triggered when a member deletes its account.
flag.created Someone flags a problem for admin attention
local_chapter.last_organiser.deleted A local chapter’s last organiser leaves the group
local_chapter.organiser_request.created A user applies to be a local chapter organiser
member.deleted A member is deleted
member.deleted.resources_transferred Resources previously owned by a deleted member have been transferred to another
petition.flagged A petition is flagged for the first or fifth time
petition.inappropriate.creator_message An inappropriate petition’s creator writes a message to admins
petition.launched A new petition is launched
petition.launched.ham A petition passes the post-launch spam check
petition.launched.requires_moderation A new petition requires moderation
petition.reactivated A hidden or ended petition is reactivated
petition.updated A petition is updated
petition.updated.requires_moderation An updated petition requires moderation
petition.target.response Decision maker responded to a petition notification
user.created New user created.
user.confirmed User has confirmed its new account. This event is only emitted for organisations who have user confirmation turned on.
user.changed A user has changed their saved first_name, last_name, postcode or locale.
signature.created Member signs a petition
signature.deleted Signature permanently deleted. Signature deletions are distinct from unsubscribes and should be treated as if the signature never happened
signature.confirmed
unsubscribe.created Member unsubscribes. This notification is emitted when member: is globally unsubscribed from the organisation by an admin or through the API, unsubscribes from partnership or unsubscribes from a petition.
locale.created New i18n instance created
forum.message.requires_moderation A new message requires moderation

blast_email.created

Example payload for blast_email.created:

  {
  "type": "blast_email.created",
  "data": {
    "id": 100,
    "from_name": "Jane Doe",
    "from_address": "jane@example.com",
    "subject": "Please help spread the word",
    "recipient_count": 990
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A new blast email is ready for moderation

category.created

Example payload for category.created:

  {
  "type": "category.created",
  "data": {
    "id": 22,
    "name": "Environment",
    "slug": "environment",
    "external_id": null,
    "created_at": "2015-08-03T12:38:46-04:00",
    "updated_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "df67126973794c2efde76cc4"
}

A new category is created

category.updated

Example payload for category.updated:

  {
  "type": "category.updated",
  "data": {
    "id": 22,
    "name": "Environment",
    "slug": "environment",
    "external_id": null,
    "created_at": "2015-08-03T12:38:46-04:00",
    "updated_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "df67126973794c2efde76cc4"
}

A category is changed

category.deleted

Example payload for category.deleted:

  {
  "type": "category.deleted",
  "data": {
    "id": 22,
    "name": "Environment",
    "slug": "environment",
    "external_id": null,
    "created_at": "2015-08-03T12:38:46-04:00",
    "updated_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "df67126973794c2efde76cc4"
}

A category has been deleted and removed from the organisation

data.full_table_exported

Example payload for data.full_table_exported:

  {
  "type": "data.full_table_exported",
  "data": {
    "url": "https://agra-data-exports.s3.amazonaws.com/default/petitions_20150803180434.csv?AWSAccessKeyId=XXX&Expires=1438711490&Signature=ABCDE%3D",
    "table": "petitions"
  },
  "jid": "cf0f50d7d225e20a188be328"
}

A full-table dump is ready for retrieval at S3

Note URL is encoded using (7-bit) ASCII and some characters will be escaped (i.e.: ’&’, ’<’, ’>’, etc.). See http://www.rubydoc.info/gems/activesupport/3.2.22/ActiveSupport/JSON/Encoding for more details

data.incremental_table_exported

Example payload for data.incremental_table_exported:

  {
  "type": "data.incremental_table_exported",
  "data": {
    "url": "https://agra-data-exports.s3.amazonaws.com/default/petitions_20150803180434.csv?AWSAccessKeyId=XXX&Expires=1438711490&Signature=ABCDE%3D",
    "table": "petitions"
  },
  "jid": "cf0f50d7d225e20a188be328"
}

An incremental CSV update is ready for retrieval at S3

Note URL is encoded using (7-bit) ASCII and some characters will be escaped (i.e.: ’&’, ’<’, ’>’, etc.). See http://www.rubydoc.info/gems/activesupport/3.2.22/ActiveSupport/JSON/Encoding for more details

event.created

Example payload for event.created:

  {
  "type": "event.created",
  "data": {
    "slug": "petition-hand-in",
    "title": "Petition Hand In",
    "url": "https://demo.controlshiftlabs.com/events/petition-hand-in"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A new event is published

event.updated

Example payload for event.updated:

  {
  "type": "event.updated",
  "data": {
    "slug": "petition-hand-in",
    "title": "Petition Hand In",
    "url": "https://demo.controlshiftlabs.com/events/petition-hand-in"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

An event is updated

attendee.created

Example payload for attendee.created:

  {
  "type": "attendee.created",
  "data": {
    "id": 645,
    "first_name": "Jennifer",
    "last_name": "Goines",
    "email": "jenny@example.com",
    "phone_number": null,
    "postcode": "23456",
    "country": "",
    "attending_status": "attending",
    "join_organisation": null,
    "join_group": null,
    "user_ip": "127.0.0.1",
    "additional_fields": {
    },
    "member": {
      "id": 747,
      "created_at": "2015-08-03T13:30:01-04:00"
    },
    "event": {
      "url": "http://localhost/events/meetup-at-park",
      "slug": "meetup-at-park"
    },
    "created_at": "2015-08-03T13:30:01-04:00"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

Member RSVP to event

attendee.updated

Example payload for attendee.updated:

  {
  "type": "attendee.updated",
  "data": {
    "id": 645,
    "first_name": "Jennifer",
    "last_name": "Goines",
    "email": "jenny@example.com",
    "phone_number": null,
    "postcode": "23456",
    "country": "",
    "attending_status": "attending",
    "join_organisation": null,
    "join_group": null,
    "user_ip": "127.0.0.1",
    "additional_fields": {
    },
    "member": {
      "id": 747,
      "created_at": "2015-08-03T13:30:01-04:00"
    },
    "event": {
      "url": "http://localhost/events/meetup-at-park",
      "slug": "meetup-at-park"
    },
    "created_at": "2015-08-03T13:30:01-04:00"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

Member updates RSVP to event

attendee.deleted

Example payload for attendee.deleted:

  {
  "type": "attendee.deleted",
  "data": {
    "id": 645,
    "first_name": "Jennifer",
    "last_name": "Goines",
    "email": "jenny@example.com",
    "phone_number": null,
    "postcode": "23456",
    "country": "",
    "attending_status": "attending",
    "join_organisation": null,
    "join_group": null,
    "user_ip": "127.0.0.1",
    "additional_fields": {
    },
    "member": {
      "id": 747,
      "created_at": "2015-08-03T13:30:01-04:00"
    },
    "event": {
      "url": "http://localhost/events/meetup-at-park",
      "slug": "meetup-at-park"
    },
    "created_at": "2015-08-03T13:30:01-04:00"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

Attendee permanently deleted. This is currently only triggered when a member deletes its account.

flag.created

Example payload for flag.created:

  {
  "type": "flag.created",
  "data": {
    "id": 7,
    "flagged": {
      "type": "ForumMessage",
      "id": 224
    },
    "flagged_by": {
      "email": "bobby@example.com",
      "name": "Bobby Singer"
    },
    "reason": "This violates our community commitments",
    "status": "unreviewed",
    "created_at": "2015-08-05T13:17:46-04:00",
    "updated_at": "2015-08-05T13:17:46-04:00"
  },
  "jid": "f848f4b38b9125f7089d59ad"
}

Someone flags a problem for admin attention

For example, a local chapter organiser flags a forum message.

local_chapter.last_organiser.deleted

Example payload for local_chapter.last_organiser.deleted:

  {
  "type": "local_chapter.last_organiser.deleted",
  "data": {
    "slug": "springfield-members",
    "name": "Springfield Members",
    "url": "https://demo.controlshiftlabs.com/local_chapters/springfield-members"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A local chapter’s last organiser leaves the group

local_chapter.organiser_request.created

Example payload for local_chapter.organiser_request.created:

  {
  "type": "local_chapter.organiser_request.created",
  "data": {
    "id": 1234,
    "local_chapter": {
      "slug": "springfield-members",
      "name": "Springfield Members",
      "url": "https://demo.controlshiftlabs.com/local_chapters/springfield-members"
    },
    "user": {
      "id": 505,
      "email": "jane@example.com",
      "full_name": "Jane Goodall",
      "first_name": "Jane",
      "last_name": "Goodall",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "reason": "I want to be part of this amazing group",
    "created_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A user applies to be a local chapter organiser

member.deleted

Example payload for member.deleted:

  {
  "type": "member.deleted",
  "data": {
    "member": {
      "id": 100,
      "email": "jane@example.com"
    },
    "resources": {
      "new_owner": {
        "id": 888,
        "email": "richard@example.com",
        "full_name": "Richard Chamberlain",
        "first_name": "Richard",
        "last_name": "Chamberlain",
        "phone_number": "12345-1",
        "postcode": "34058-2356"
      }
    }
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A member is deleted

member.deleted.resources_transferred

Example payload for member.deleted.resources_transferred:

  {
  "type": "member.deleted.resources_transferred",
  "data": {
    "member": {
      "id": 100,
      "email": "jane@example.com"
    },
    "resources": {
      "new_owner": {
        "id": 888,
        "email": "richard@example.com",
        "full_name": "Richard Chamberlain",
        "first_name": "Richard",
        "last_name": "Chamberlain",
        "phone_number": "12345-1",
        "postcode": "34058-2356"
      }
    }
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

Resources previously owned by a deleted member have been transferred to another

petition.flagged

Example payload for petition.flagged:

  {
  "type": "petition.flagged",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A petition is flagged for the first or fifth time

petition.inappropriate.creator_message

Example payload for petition.inappropriate.creator_message:

  {
  "type": "petition.inappropriate.creator_message",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

An inappropriate petition’s creator writes a message to admins

petition.launched

Example payload for petition.launched:

  {
  "type": "petition.launched",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A new petition is launched

petition.launched.ham

Example payload for petition.launched.ham:

  {
  "type": "petition.launched.ham",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A petition passes the post-launch spam check

petition.launched.requires_moderation

Example payload for petition.launched.requires_moderation:

  {
  "type": "petition.launched.requires_moderation",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A new petition requires moderation

petition.reactivated

Example payload for petition.reactivated:

  {
  "type": "petition.reactivated",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A hidden or ended petition is reactivated

petition.updated

Example payload for petition.updated:

  {
  "type": "petition.updated",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A petition is updated

petition.updated.requires_moderation

Example payload for petition.updated.requires_moderation:

  {
  "type": "petition.updated.requires_moderation",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

An updated petition requires moderation

petition.target.response

Example payload for petition.target.response:

  {
  "type": "petition.target.response",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "locale": "en-US",
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

Decision maker responded to a petition notification

user.created

Example payload for user.created:

  {
  "type": "user.created",
  "data": {
    "id": 180,
    "email": "jenny@example.com",
    "fullname": "Jennifer Goines",
    "first_name": "Jennifer",
    "last_name": "Goines",
    "phone_number": "123-456",
    "postcode": "101010",
    "country": "US",
    "confirmed_at": "2015-08-03T13:30:01-04:00",
    "join_organisation": true
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

New user created.

user.confirmed

Example payload for user.confirmed:

  {
  "type": "user.confirmed",
  "data": {
    "id": 180,
    "email": "jenny@example.com",
    "fullname": "Jennifer Goines",
    "first_name": "Jennifer",
    "last_name": "Goines",
    "phone_number": "123-456",
    "postcode": "101010",
    "country": "US",
    "confirmed_at": "2015-08-03T13:30:01-04:00",
    "join_organisation": true
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

User has confirmed its new account. This event is only emitted for organisations who have user confirmation turned on.

user.changed

Example payload for user.changed:

  {
  "type": "user.changed",
  "data": {
    "id": 180,
    "email": "foo@bar.com",
    "changes": {
      "first_name": [
        "Nate",
        "Nathan"
      ]
    }
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A user has changed their saved first_name, last_name, postcode or locale.

signature.created

Example payload for signature.created:

  {
  "type": "signature.created",
  "data": {
    "id": 645,
    "first_name": "Jennifer",
    "last_name": "Goines",
    "email": "jenny@example.com",
    "phone_number": null,
    "postcode": "23456",
    "country": "",
    "country_without_fallback": "",
    "join_organisation": null,
    "join_group": null,
    "locale": "en",
    "token": "2919ec5f25d493c19690a21135d7fd902a336927",
    "source": "",
    "bucket": "",
    "user_ip": "127.0.0.1",
    "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36",
    "daisy_chain_used": null,
    "from_embed": null,
    "confirmed_at": null,
    "confirmed_reason": null,
    "last_signed_at": "2015-08-03T13:30:01-04:00",
    "additional_fields": {
    },
    "member": {
      "id": 747,
      "created_at": "2015-08-03T13:30:01-04:00"
    },
    "petition": {
      "url": "http://localhost/petitions/don-t-destroy-our-park",
      "slug": "don-t-destroy-our-park"
    },
    "created_at": "2015-08-03T13:30:01-04:00",
    "updated_at": "2015-08-03T13:30:01-04:00"
  },
  "jid": "1ebf2eb10c3d8939f42dff68"
}

Member signs a petition

We also include optional partnership or effort slugs nested under the petition resource for signatures that take place on petitions associated with those objects.

signature.deleted

Example payload for signature.deleted:

  {
  "type": "signature.created",
  "data": {
    "id": 645,
    "first_name": "Jennifer",
    "last_name": "Goines",
    "email": "jenny@example.com",
    "phone_number": null,
    "postcode": "23456",
    "country": "",
    "country_without_fallback": "",
    "join_organisation": null,
    "join_group": null,
    "locale": "en",
    "token": "2919ec5f25d493c19690a21135d7fd902a336927",
    "source": "",
    "bucket": "",
    "user_ip": "127.0.0.1",
    "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36",
    "daisy_chain_used": null,
    "from_embed": null,
    "confirmed_at": null,
    "confirmed_reason": null,
    "last_signed_at": "2015-08-03T13:30:01-04:00",
    "additional_fields": {
    },
    "member": {
      "id": 747,
      "created_at": "2015-08-03T13:30:01-04:00"
    },
    "petition": {
      "url": "http://localhost/petitions/don-t-destroy-our-park",
      "slug": "don-t-destroy-our-park"
    },
    "created_at": "2015-08-03T13:30:01-04:00",
    "updated_at": "2015-08-03T13:30:01-04:00"
  },
  "jid": "1ebf2eb10c3d8939f42dff68"
}

Signature permanently deleted. Signature deletions are distinct from unsubscribes and should be treated as if the signature never happened

We also include optional partnership or effort slugs nested under the petition resource for signatures that take place on petitions associated with those objects.

signature.confirmed

Example payload for signature.confirmed:

  {
  "type": "signature.created",
  "data": {
    "id": 645,
    "first_name": "Jennifer",
    "last_name": "Goines",
    "email": "jenny@example.com",
    "phone_number": null,
    "postcode": "23456",
    "country": "",
    "country_without_fallback": "",
    "join_organisation": null,
    "join_group": null,
    "locale": "en",
    "token": "2919ec5f25d493c19690a21135d7fd902a336927",
    "source": "",
    "bucket": "",
    "user_ip": "127.0.0.1",
    "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36",
    "daisy_chain_used": null,
    "from_embed": null,
    "confirmed_at": null,
    "confirmed_reason": null,
    "last_signed_at": "2015-08-03T13:30:01-04:00",
    "additional_fields": {
    },
    "member": {
      "id": 747,
      "created_at": "2015-08-03T13:30:01-04:00"
    },
    "petition": {
      "url": "http://localhost/petitions/don-t-destroy-our-park",
      "slug": "don-t-destroy-our-park"
    },
    "created_at": "2015-08-03T13:30:01-04:00",
    "updated_at": "2015-08-03T13:30:01-04:00"
  },
  "jid": "1ebf2eb10c3d8939f42dff68"
}

We also include optional partnership or effort slugs nested under the petition resource for signatures that take place on petitions associated with those objects.

unsubscribe.created

Example payload for unsubscribe.created:

  [
  {
    "type": "unsubscribe.created",
    "data": {
      "id": 3,
      "email": "river@example.com",
      "unsubscribe_organisation": true,
      "unsubscribable_type": null,
      "created_at": "2015-08-03T13:38:49-04:00"
    },
    "jid": "030ee43a4fa9788e084d9cfc"
  },
  {
    "type": "unsubscribe.created",
    "data": {
      "id": 3,
      "email": "river@example.com",
      "unsubscribe_organisation": false,
      "unsubscribable_type": "GroupSubscription",
      "created_at": "2015-08-03T13:38:49-04:00"
    },
    "jid": "030ee43a4fa9788e084d9cfc"
  },
  {
    "type": "unsubscribe.created",
    "data": {
      "id": 3,
      "email": "river@example.com",
      "unsubscribe_organisation": true,
      "unsubscribable_type": "Signature",
      "created_at": "2015-08-03T13:38:49-04:00",
      "blast_email": {
        "id": 199,
        "subject": "Please help spread the word"
      }
    },
    "jid": "030ee43a4fa9788e084d9cfc"
  }
]

Member unsubscribes. This notification is emitted when member: is globally unsubscribed from the organisation by an admin or through the API, unsubscribes from partnership or unsubscribes from a petition.

First example: Admin globally unsubscribes a member from the member page

Second example: Member unsubscribes from partnership
Note: unsubscribe_organisation can have true or false values depending on member’s input at the unsubscribe page (only if the Allow global unsubscribes option is enabled for the organisation)

Third example: Member unsubscribes from petition
Notes:
unsubscribe_organisation can have true or false values depending on member’s input at the unsubscribe page (only if the Allow global unsubscribes option is enabled for the organisation)
The blast_email property will only be included if member unsubscribed by clicking on the link included on the petition blast email’s footer

locale.created

Example payload for locale.created:

  {
  "type": "locale.created",
  "data": {
    "id": 22,
    "name": "es",
    "action_kit_external_id": null,
    "created_at": "2015-08-03T12:38:46-04:00",
    "updated_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "df67126973794c2efde76cc4"
}

New i18n instance created

forum.message.requires_moderation

Example payload for forum.message.requires_moderation:

  {
  "type": "forum.message.requires_moderation",
  "data": {
    "id": 10,
    "thread_title": "Hello everyone",
    "content": "I just wanted to say hi",
    "admin_status": "unreviewed",
    "sent_at": "2015-08-05T13:17:46-04:00",
    "member": {
      "id": 10829,
      "created_at": "2015-08-05T13:17:46-04:00"
    },
    "discussable": {
      "discussable_type": "LocalChapter",
      "discussable_id": 56
    }
  },
  "jid": "f848f4b38b9125f7089d59ad"
}

A new message requires moderation

Note
The admin_status field can take any of the following values:
  • unreviewed
  • flagged
  • approved_automatically
  • approved_manually
  • flagged_deleted
  • inappropriate
Though on the forum.message.requires_moderation notification type it will always be set to unreviewed
The discussable.discussable_type field can take any of the following values:
  • LocalChapter
  • Event
  • CalendarEvent
  • LocalChapterEvent
  • PetitionEvent

Bulk Data

ControlShift’s Bulk Data Webhooks make it easy to pull your data into external services.

Webhooks

We provides two special bulk data webhooks to help you keep an external reporting or analytics database server up to date with information from ControlShift’s internal tables. The data.full_table_exported and data.incremental_table_exported webhooks can be consumed to keep an external database mirror containing ControlShift data up to date. This service was built in a database agnostic way, but it should be possible to build a ControlShift -> Amazon Redshift data pipeline using the ControlShift to Redshift Pipeline technique outlines below.

Bulk Data Data Schemas

The bulk data webhooks include exports of the following tables. You can use the links to download CSV headers or a psql schema for each table.

Additional files:

ControlShift to Redshift Pipeline

Setting up an Amazon Redshift integration should take one to two hours. We’ll use a custom AWS Lamda to receive data from ControlShift’s webhook and Amazon’s Lamda Redshift Loader to load our data into Redshift. For this example, we’ll focus on the daily full_table dump, and truncating the previous days data for petitions only. You could also add a second pipeline to add incremental data if you need more frequent updates.

As of this writing, aws-lambda-redshift-loader v2.4.0 only supports a single configuration per Lambda. This means you’ll have to setup a separate Lambda for every table you want to capture. If you want to capture both nightly and incremental dumps, you’ll also need to set up one nightly lambda and one incremental lambda per table you want to capture. It can be a little tedious, but remember, the first one is the hardest!

Example Flow

ControlShift Redshift Flow Chart

Redshift Pipeline Setup

The core steps to get this up and running are:

  1. Setting up S3 buckets to retain file storage
  2. Setting up your custom lambda webhook receiver
  3. Setting up your Redshift database cluster and pre-loading a schema.
  4. Setting up your aws-lambda-redshift-loader lambda.
  5. Configure your bulk data webhook in the Control Shift settings console.

Note: All resources should be set up in the same AWS region (e.g.: us-east-1) so they can access each other. “US Standard” and “us-east-1” are the same thing.

Before starting: You’ll need git, npm, and access to a PostgreSQL connection tool, such as psql on the command line (Redshift is based on PostgreSQL).

These instructions were prepared for aws-lambda-redshift-loader v2.4.0.


Setting Up S3 Buckets

  1. Login to you AWS console and create two s3 buckets for future use.
    1. controlshift-receiver - will hold received csv files
    2. controlshift-manifests - will hold resulting manifest files from your aws-lambda-redshift-loader
  2. That’s it. You’re done here.

Setting Up Custom Lambda Receiver

Lambda Receiver Code

var https = require('https');
var url = require('url');
var AWS = require('aws-sdk');

function csvReceiver(downloadUrl, eventContext, webhookJSON){
  var targetBucket = 'controlshift-receiver', // receiver bucket name
    s3 = new AWS.S3(),
    firstLineRemoved = false,
    csvData = '',
    context = eventContext,
    receivedJSON = webhookJSON;

  var parseFileName = function(url_string){
    path = url.parse(url_string).pathname
    return path.substr(path.lastIndexOf('/')+1)
  }

  // Remove Headers
  var removeFirstLine = function(chunk){
    if(firstLineRemoved){
      csvData += chunk;
    }else{
      var buffer = chunk.toString();
      if (buffer.indexOf('\n') !== -1) {
        csvData += chunk.slice(chunk.indexOf('\n') + 1);
        firstLineRemoved = true;
        buffer = null;
      }
    }
  }

  var uploadCsv = function(){
    var upload_details = {Bucket: targetBucket, Key: fileName, Body: csvData};
    s3.upload(upload_details, function(err, data) {
      if (err){
        // an error occurred
        console.log(err, err.stack);
        context.fail(err);
      }else{
        // successful response
        context.succeed({"status": "success", "payload": receivedJSON});
      }
    });
  }

  return {
    processCsv: function(){
      fileName = parseFileName(downloadUrl);
      https.get(downloadUrl, function(httpResponse) {
        httpResponse.on('data', function(chunk) {
          removeFirstLine(chunk);
        });
        httpResponse.on('end', function() {
          console.log(httpResponse);
          uploadCsv();
        });
      });
    }
  }
}

// Lambda event Handler
exports.handler = function(event, context) {
  var receivedJSON = JSON.stringify(event, null, 2);
  console.log('Received event:', receivedJSON);
  if(event.type == 'data.full_table_exported'){
    var receiver = new csvReceiver(event.data.url, context, receivedJSON);
    receiver.processCsv();
  }else{
    context.succeed({"status": "skipped", "payload": receivedJSON});
  }
};

From the Lambda management console, you’ll need to do the following.

Create the Receiver Lambda

  1. Click to create a new Lambda function.
  2. Skip the existing blueprints (button towards the bottom of page)
  3. Name your function something like “controlShiftWebhookReciever” and leave the runtime to node default.
  4. Copy and paste the “Lambda Receiver” function from the right into the inline lambda code editor.
  5. Update the targetBucket in the inline code editor (approximately line 6 after copying/pasting).
  6. Leave the handler as the default.
  7. For role, select “S3 Execution role”. This will open a new window.
  8. In this new window, called “IAM Management Console”, the IAM role field should say “Create a new role.” Name this new role something relevant, like “controlShiftReceriver_s3.” Then click “Allow” in the far bottom left corner, this returns you to the previous window.
  9. Next, set the memory size to 128MB. If you have a large petitions DB, you will likely need more. You can always update the memory size later and you can check the log stream in CloudWatch monitoring to see how much memory was used.
  10. Set the Timeout a little higher - 30 seconds should be fine.
  11. Use “No VPC”.
  12. Click next, then on the next page just click “Create Function”!

Setup the API Endpoint

Now that we have our Lambda setup, we need to be able to POST to it from your Control Shift instance.

  1. If not already there, go back to the Receiver Lambda we just created.
  2. Click on the “API Endpoints” tab and click to add a new endpoint.
  3. Select an API Gateway.
  4. Set the Method to POST and Security to Open.
  5. Click create. Done!

Setting Up RedShift Database

Click on Redshift service in the AWS console.

Launch Your Cluster

  1. Click to create your cluster.
  2. Setup a identifier, database name, master user name and password as you desire and click next. Let’s use these values for this example:
    • Cluster: control-shift-cluster
    • Database name: controlshiftdb
    • Port: Leave the default
    • Master Username: masteruser
  3. Use the defaults on the next two screens, “Node Configuration” and “Additional Configuration, just click next.
  4. Then Launch!
  5. Once the creation and launch process finishes, you can click to view your new cluster’s properties, which will include an endpoint address. Save that address somewhere. Example: control-shift-cluster.xXxXxX00Xx.us-east-1.redshift.amazonaws.com

Prepare Redshift Connection Security

We need to whitelist relevant IP addresses in our default security group. We’ll need to add two rules. While we’re managing our security through a VPC (Virtual Private Cloud), the best UI for our purposes is actually in the EC2 configuration console.

  1. Go to the AWS Console > EC2
  2. Click on "Security Groups” in the left hand menu.
  3. Click on the default Security Group (it should say default in the “Group Name” column)
  4. Click on the “Inbound Rules” tab at the bottom and click “Edit.”
  5. Click “Add Rule” to start adding our first rule.
  6. Select “All Traffic” for “type” and for “source” select “My IP.” This should pre-fill your CIDR. Save the new rule.
  7. Click “Add Rule” again.
  8. Select “Redshift” for “Type.” Enter your cluster’s port (typically 5439). Type “sg-” into the source field, and it should pull up your default security group.

Note: If you’re having trouble getting your own IP/CIDR setup, try either looking up your subnet mask and using this table. Your CIDR will almost always be your ip address followed by /32 or /64 for ipv6. Example: 123.123.123.123/32. If all else fails, use the “Anywhere” global option 0.0.0.0/0 - opening your cluster to the public Internet. The latter isn’t recommended. If you do have to do this, make sure you remove the rule when you’re done.

Prepare Redshift Schema

We need to prep our tables data schema to receive our ControlShift Data. We’ll use psql for this.

  1. Install psql if you don’t have it already.
  2. Download the current schema for our petitions table as petitions_schema.sql.
  3. Finally, you’ll need to load that schema into your table with the following command:

psql -h <cluster_endpoint> -U <database_user> -d <database_name> -p <cluster_port> -f petitions_schema.sql

Note: When connecting, you should almost immediately get a request for your password. If your connection is timing out before you enter your password, there is an authorization issue. If you are timing out after connecting, you can extend your keep alive timeout with the following shell command.

sudo /sbin/sysctl -w net.ipv4.tcp_keepalive_time=200 net.ipv4.tcp_keepalive_intvl=200 net.ipv4.tcp_keepalive_probes=5

Creating a Database User & Granting Privileges

Creating a user and granting INSERT Privileges

CREATE USER redshift_user PASSWORD PlainTextPassword123;
ALTER TABLE petitions OWNER TO redshift_user;
GRANT INSERT ON petitions TO redshift_user;

We need to allow our user to access our aws-lambda-redshift-loader to access our Redshift database. Connect via psql and create your user and grant permissions. See code example.

psql -h <cluster_endpoint> -U <database_user> -d <database_name> -p <cluster_port>

Security Notes:


Setting up aws-lambda-redshift-loader

Deploying the aws-lambda-redshift-loader Function

AWS Lambda Execution Role

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"Stmt1424787824000",
      "Effect":"Allow",
      "Action":[
        "dynamodb:DeleteItem",
        "dynamodb:DescribeTable",
        "dynamodb:GetItem",
        "dynamodb:ListTables",
        "dynamodb:PutItem",
        "dynamodb:Query",
        "dynamodb:Scan",
        "dynamodb:UpdateItem",
        "sns:GetEndpointAttributes",
        "sns:GetSubscriptionAttributes",
        "sns:GetTopicAttributes",
        "sns:ListTopics",
        "sns:Publish",
        "sns:Subscribe",
        "sns:Unsubscribe",
        "s3:Get*",
        "s3:Put*",
        "s3:List*",
        "kms:Decrypt",
        "kms:DescribeKey",
        "kms:GetKeyPolicy"
      ],
      "Resource":[
        "*"
      ]
    },
    {
      "Effect":"Allow",
      "Action":[
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource":"arn:aws:logs:*:*:*"
    },
    {
      "Effect":"Allow",
      "Action":[
        "ec2:CreateNetworkInterface",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DetachNetworkInterface",
        "ec2:DeleteNetworkInterface"
      ],
      "Resource":"*"
    }
  ]
}
  1. Go to the AWS Lambda Console in the same region as your S3 bucket and Amazon Redshift cluster.
  2. Select Create a Lambda function and enter a name (e.g.: controlShiftRedshiftLoader).
  3. Clone the Amazon Redshift Database Loader repository locally
  4. Under Code entry type select ‘Upload a zip file’ and upload the AWSLambdaRedshiftLoader-2.4.0.zip file from the “dist” directory on the repository you just cloned.
  5. Set the Runtime to Node.js 0.10.
  6. Use the default values for the handler, and in the Role drop-down, select “* Basic Execution Role.” A IAM creation wizard will open in a new window.
  7. Follow the wizard, selecting to “Create a new IAM Role” and name it as you like. Click to “View the Policy” and then click edit.
  8. Copy and paste the the AWS Lambda Execution Role permissions from the right.
  9. Then click “Add Policy.” Select the AmazonRedshiftFullAccess role and add it, then add another role - AmazonDMSRedshiftS3Role.
  10. Navigate back to your Lambda setup tab and set the max timeout (5 minutes) to accommodate potentially long COPY times.
  11. From the VPC dropdown, select your default VPC.
  12. Leave the rest of the settings unchanged.
  13. Click next and then click to create your Lambda.

Establishing an Event Source

Once deployed, you need to add an event source:

  1. Select “Event Source” tab and click to “Add event source.”
  2. The event source type should be set to S3.
  3. Select the S3 bucket you want to use for input data - controlshift-receiver.
  4. Select the 'Object Created (All)’ notification type.

Configuration Prep

Setup your AWS credentials as environment variables as follows: export AWS_ACCESS_KEY_ID=XXXXXXXXX AWS_SECRET_ACCESS_KEY=XXXXXXXXXXX AWS_REGION=us-east-1

Running setup.js

  1. After setting up your environment, head to your local copy of aws-lambda-redshift-loader.
  2. run npm install to install package’s dependencies.
  3. run node setup.js.
  4. The script will request various configuration options. Some sensible responses for our use case are below. Empty responses were just skipped.
Enter the Region for the Configuration > us-east-1
Enter the S3 Bucket & Prefix to watch for files > controlshift-receiver
Enter a Filename Filter Regex >
Enter the Cluster Endpoint > control-shift-cluster.xXxXxX00Xx.us-east-1.redshift.amazonaws.com
Enter the Cluster Port > 5439
Does your cluster use SSL (Y/N) > Y
Enter the Database Name > controlshiftdb
Enter the Table to be Loaded > petitions
Enter the comma-delimited column list (optional) > (Copy and paste from our petitions_headers.csv)
Should the Table be Truncated before Load? (Y/N) > Y # Don’t truncate if this lambda will handle incremental builds.
Enter the Database Username > masteruser # Or any user you created in psql
Enter the Database Password > SuPeRs8CuR1tY # your password
Enter the Data Format (CSV, JSON or AVRO) > CSV
Enter the CSV Delimiter > ,
Enter the S3 Bucket for Redshift COPY Manifests > controlshift-manifests
Enter the Prefix for Redshift COPY Manifests > manifests
Enter the Prefix to use for Failed Load Manifest Storage > failed
Enter the Access Key used by Redshift to get data from S3. If NULL then Lambda execution role credentials will be used >
Enter the Secret Key used by Redshift to get data from S3. If NULL then Lambda execution role credentials will be used >
Enter the SNS Topic ARN for Successful Loads >
Enter the SNS Topic ARN for Failed Loads >
How many files should be buffered before loading? > 1
How old should we allow a Batch to be before loading (seconds)? > 120
Additional Copy Options to be added >
If Encrypted Files are used, Enter the Symmetric Master Key Value >

Configure ControlShift’s webhook

Finally, you’ll need to log into your admin panel. Settings > CRM Integrations & Webhooks > Configure Webhook Endpoints. Then add a bulk data webhook pointing at your Lambda endpoint.

Troubleshooting Errors

Example LambdaRedshiftBatches error

{
  "control-shift-cluster.ckcgajpfnj82.us-east-1.redshift.amazonaws.com":{
    "status":-1,
    "error":{
      "name":"error",
      "length":143,
      "severity":"FATAL",
      "code":"28000",
      "file":"/home/awsrsqa/padb/src/pg/src/backend/libpq/auth.c",
      "line":"402",
      "routine":"auth_failed"
    }
  }
}

A few tips if things aren’t working:

Authenticated REST API

The Authenticated REST API allows customers to build applications that interact with their own data or securely grant access to third-party app developers without exposing administrative credentials. It is designed to be consumed server-side, in contrast to the JSONP API which is designed for unauthenticated javascript integrations.

We use OAuth2 to allow organizations to securely grant access to API Applications. OAuth2 is a standardized protocol that allows user credentials to be exchanged for a secret token which can then be used to access an API. There are OAuth2 implementations available for most common programming languages. You’ll probably want to read some general documentation on how OAuth2 works before attempting to use this API.

An API Application must be configured within your ControlShift instance before using the API. Access is then granted to this application, and exchanged for a token which is used to authenticate your access to API endpoints.

Quickstart Guide

These are the steps to setup a new API Application and to use that new API Application to access platform resources through OAuth2. In a production context, automating these steps is recommended.

  1. Setup a new API Application Login as an Organisation Owner and navigate to Settings > Integrations > Configure API Applications. Add a New Application. Use urn:ietf:wg:oauth:2.0:oob as the callback URL unless you plan to allow self-service application authorization.

  2. Generate Authorization Code From the API Application show page, click “Authorize”. This will generate a code that can be exchanged with a token to act as the currently signed in user. After the authorization is approved, an authorization code should be displayed. Record this code for the next step.

Use cURL to do the token exchange

curl -X POST --header "Content-Type: application/json" --header "Accept: application/json" --data '{"grant_type":"authorization_code", "code":"{code}", "client_id":"{client_id}", "client_secret":"{client_secret}", "redirect_uri":"{redirect_uri}"}' https://foo.controlshiftlabs.com/oauth/token
  1. Exchange Authorization Code for Token Take the authorization code from the previous step and exchange it for an OAuth2 Token.

POST https://foo.controlshiftlabs.com/oauth/token client_id=... redirect_uri=... grant_type=authorization_code client_secret=... code=...

  1. Use Token to Access API The token can now be used to access API endpoints as the authorized user.

GET https://foo.controlshiftlabs.com/api/v1/organisation?access_token={token}

Should return a json representation of the current organisation. See documentation for other endpoints that are available.

We’ve also written an example of this same work flow in ruby using the oauth2 gem.

In most production scenarios instead of manually performing the authorization, and exchanging the code for a token, this setup process would be automated. See your oauth2 library documentation for more information.

Organisation

Retrieve

Get up to date statistics about the current organisation.

GET /api/v1/organisation

{
  "organisation": {
    "slug": "foo",
    "petitions_count": 3,
    "signatures_count": 4,
    "blast_emails_count": 0,
    "members_count": 5
  }
}

Members

The platform transparently creates a member record for any email address that creates a user account, signs a petition, or attends an event. Members are unique by email address, and all platform activity is tracked by member id.

Lookup

{
  "member": {
    "id": 123,
    "email": "foo@bar.com",
    ...
  }
}

Find a member by email address. Once you have obtained a member id you can use this identifier in other API calls.

GET /api/v1/members/lookup?email=foo@bar.com

Show

Find a member by id. Member ids can be discovered via a call to the lookup API.

GET /api/v1/members/123

Destroy

{
  "member": {
    "deleted": true, 
    "id": 123,
    "email": "foo@bar.com"
  }
}

Permanently deletes a member with specified id and all associated records including petitions, signatures, events. The deletion feature is designed to honor requests from members around the right to be forgotten.

DELETE /api/v1/members/123

Deletes are synchronous and may take several seconds depdending on how many resources are owned by the member.

Ownership of Petitions and Events created by the deleted member will be re-assigned to the user account specified in the organisation’s settings.

Unsubscribe

{
  "member": {
    "unsubscribed": true, 
    "id": 123,
    "email": "foo@bar.com"
  }
}

Unsubscribes a member with specified id from emails from event hosts and petiton creators. Member will still receive transactional emails, and will be resubscribed if they opt in while signing a new petition or attending a new event, etc.

POST /api/v1/members/123/unsubscribe