Create IPUMS IHGIS Data Extracts
Below we provide examples in curl showing how to create and manage IHGIS data extracts with the IPUMS API.
Get your key from https://account.ipums.org/api_keys. Make sure to replace ‘MY_KEY’ (all caps) in the snippet below with your key.
Load Libraries and Set Key
export MY_KEY=MY_KEY # set the MY_KEY environment variable using bash shell
Submit a Data Extract Request
To submit a data extract request you need to pass a valid JSON-formatted extract request in the body of your POST. The names to use for values in the data extract request can be discovered via our metadata API endpoints.
Data Extract Request Fields
datasets
: An object where each key is thename
of the requested dataset and each value is another object describing your selections for that dataset.dataTables
: (Required) A list of selected data table names.tabulationGeographies
: (Required) A list of selected tabulation geography names.
description
: A short description of your extract.
curl -X POST \
"https://api.ipums.org/extracts/?collection=ihgis&version=2" \
-H "Content-Type: application/json" \
-H "Authorization: $MY_KEY" \
-d '
{
"datasets": {
"TG2010pop": {
"dataTables": [
"TG2010pop.AAA",
"TG2010pop.AAB"
],
"tabulationGeographies": [
"TG2010pop.g0"
]
}
},
"description": "example extract request"
}
'
# returns a JSON response like:
{
"number": 1,
"status": "queued",
"email": "my email",
"downloadLinks": {},
"extractDefinition": {
"description": "example extract request",
"datasets": {
"TG2010pop": {
"dataTables": [
"TG2010pop.AAA",
"TG2010pop.AAB"
],
"tabulationGeographies": [
"TG2010pop.g0"
]
}
},
"version": 2,
"collection": "ihgis"
}
}
Get a Request’s Status
After submitting your extract request, you can use the extract number to retrieve the request’s status. Here we’re retrieving status for extract number 1.
curl -X GET "http://api.ipums.org/extracts/1?collection=ihgis&version=2" -H "Authorization: $MY_KEY"
## response:
{
"number": 1,
"status": "completed",
"email": "my email",
"downloadLinks": {
"tableData": {
"url": "https://api.ipums.org/downloads/ihgis/api/v1/extracts/999999/ihgis0001.zip",
"bytes": 5826,
"sha256": null
}
},
"extractDefinition": {
"description": "example extract request",
"datasets": {
"TG2010pop": {
"tabulationGeographies": [
"TG2010pop.g0"
],
"dataTables": [
"TG2010pop.AAA",
"TG2010pop.AAB"
]
}
},
"version": 2,
"collection": "ihgis"
}
}
You will get a status such as queued
, started
, produced
canceled
, failed
or completed
.
Retrieving Your Extract
To retrieve a completed extract (using extract number 1 as the example again):
- Using the request status query above, wait until the status is
completed
. - Extract the download URL from the response, which is in the
downloadLinks
attribute:
curl -X GET \
"https://api.ipums.org/extracts/1?collection=ihgis&version=2" \
-H "Authorization: $MY_KEY"
The downloadLinks portion of the response will look like:
"downloadLinks": {
"tableData": {
"url": "https://api.ipums.org/downloads/ihgis/api/v1/extracts/999999/ihgis0001.zip",
"bytes": 5826,
"sha256": null
}
},
Next, retrieve the file(s) from the URL. You will need to pass the Authorization header with your API key to the server in order to download the data.
curl -H "Authorization: $MY_KEY" "https://api.ipums.org/downloads/ihgis/api/v1/extracts/999999/ihgis0001.zip" > mydata.zip
Now you are ready for further processing and analysis as you desire.
Get a Listing of Recent Extract Requests
You may also find it useful to get a historical listing of your extract requests. If you omit an extract number in your API call, by default this will return the 10 most recent extract requests. To adjust the amount returned, you may optionally specify a ?limit=##
parameter to get the ## most recent extracts instead.
curl -X GET \
"https://api.ipums.org/extracts?collection=ihgis&version=2" \
-H "Authorization: $MY_KEY"