Create IPUMS International Data Extracts

Below we provide examples in curl showing how to work with our IPUMS API to create and manage IPUMS International data extracts. Our ipumspy and ipumsr client SDK libraries do not yet contain support for the IPUMS International data collection; we will add SDK code samples once they do.

Get your key from your IPUMS user account management page at https://account.ipums.org/api_keys.

Load Libraries and Set Key

# The IPUMS International data collection is not yet supported in the ipumspy library.

# The IPUMS International data collection is not yet supported in the ipumsr library.

# set the IPUMS_API_KEY environment variable using bash shell
export IPUMS_API_KEY=YOUR_API_KEY_HERE

Submit a Data Extract Request

To submit a data extract request, you will construct a JSON payload manually. Once you have your request formed, you will then submit it to the API.

The names to use for samples and variables in the data extract request can be discovered on our website.

# The IPUMS International data collection is not yet supported in the ipumspy library.

# The IPUMS International data collection is not yet supported in the ipumsr library.

# construct the JSON payload manually and submit it 
curl --location --request POST 'https://api.ipums.org/extracts?collection=ipumsi&version=2' \
--header "Authorization: $IPUMS_API_KEY" \
--header 'Content-Type: application/json' \
--data-raw '{
    "description": "Example extract",
    "dataStructure": { 
        "rectangular": {
            "on": "P"
        }
    },
    "dataFormat": "fixed_width",
    "samples": {
      "mx2015a": {},
      "cl2017a": {}
    },
    "variables":{
      "AGE": {},
      "SEX": {},
      "EDATTAIN": {},
      "GEOMIG1_5": {}
    }
}'

# A successful request will return a response that includes an extract number in the number attribute:

{
    "number": 1,
    "status": "queued",
    "downloadLinks": {},
    "extractDefinition": {
        "version": 2,
        "dataStructure": {
            "rectangular": {
                "on": "P"
            }
        },
        "dataFormat": "fixed_width",
        "caseSelectWho": "individuals",
        "description": "Example extract",
        "samples": {
            "mx2015a": {},
            "cl2017a": {}
        },
        "variables": {
            "COUNTRY": {
                "preselected": true
            },
            "YEAR": {
                "preselected": true
            },
            "SAMPLE": {
                "preselected": true
            },
            "SERIAL": {
                "preselected": true
            },
            "HHWT": {
                "preselected": true
            },
            "PERNUM": {
                "preselected": true
            },
            "PERWT": {
                "preselected": true
            },
            "AGE": {},
            "SEX": {},
            "EDATTAIN": {},
            "GEOMIG1_5": {}
        },
        "collection": "ipumsi"
    }
}

You can also submit hierarchical extracts (not yet supported by the ipumspy or ipumsr client libraries).

# The IPUMS International data collection is not yet supported in the ipumspy library.

# The IPUMS International data collection is not yet supported in the ipumspy library.

# construct the JSON payload manually and submit it
curl --location --request POST 'https://api.ipums.org/extracts?collection=ipumsi&version=2' \
--header "Authorization: $IPUMS_API_KEY" \
--header 'Content-Type: application/json' \
--data-raw '{
    "description": "Example hierarchical extract",
    "dataStructure": { 
        "hierarchical": {}
    },
    "dataFormat": "fixed_width",
    "samples": {
      "mx2015a": {},
      "cl2017a": {}
    },
    "variables":{
      "AGE": {},
      "SEX": {},
      "EDATTAIN": {},
      "GEOMIG1_5": {}
    }
}'

# A successful request will return a response that includes an extract number in the number attribute:

{
    "number": 2,
    "status": "queued",
    "downloadLinks": {},
    "extractDefinition": {
        "version": 2,
        "dataStructure": {
            "hierarchical": {}
        },
        "dataFormat": "fixed_width",
        "caseSelectWho": "individuals",
        "description": "Example hierarchical extract",
        "samples": {
            "mx2015a": {},
            "cl2017a": {}
        },
        "variables": {
            "RECTYPE": {},
            "COUNTRY": {
                "preselected": true
            },
            "YEAR": {
                "preselected": true
            },
            "SAMPLE": {
                "preselected": true
            },
            "SERIAL": {
                "preselected": true
            },
            "HHWT": {
                "preselected": true
            },
            "PERNUM": {
                "preselected": true
            },
            "PERWT": {
                "preselected": true
            },
            "AGE": {},
            "SEX": {},
            "EDATTAIN": {},
            "GEOMIG1_5": {}
        },
        "collection": "ipumsi"
    }
}

Checking a Request’s Status

After submitting your extract request, you can monitor the request’s status using the extract’s number.

# The IPUMS International data collection is not yet supported in the ipumspy library.

# The IPUMS International data collection is not yet supported in the ipumsr library.

curl --request GET 'https://api.ipums.org/extracts/1?collection=ipumsi&version=2' --header 'Content-Type: application/json' --header "Authorization: $IPUMS_API_KEY"

# A successful request will provide a response object like below. The exact fields may vary depending on how far along the extract is in processing.
# You will get a status such as `queued`, `started`, `produced` `canceled`, `failed` or `completed` in the status field. 

{
    "number": 1,
    "status": "completed",
    "downloadLinks": {
        "basicCodebook": {
            "url": "https://api.ipums.org/downloads/ipumsi/api/v1/extracts/1234567/ipumsi_00001.cbk",
            "bytes": 82924,
            "sha256": "eb3f5a1a2b238b08c86ed83336702bc8f86d2ca33c30c5009bcecfcde65c50cf"
        },
        "ddiCodebook": {
            "url": "https://api.ipums.org/downloads/ipumsi/api/v1/extracts/1234567/ipumsi_00001.xml",
            "bytes": 223103,
            "sha256": "e438d65dc7130656e5844e60d1565ea51ad48a250ff268f609af115a49083df1"
        },
        "stataCommandFile": {
            "url": "https://api.ipums.org/downloads/ipumsi/api/v1/extracts/1234567/ipumsi_00001.do",
            "bytes": 121969,
            "sha256": "a2c991427a56a2cd8f4301fdf8b9298324ccfffd8fdb818d0d9f3ac3b753d64c"
        },
        "data": {
            "url": "https://api.ipums.org/downloads/ipumsi/api/v1/extracts/1234567/ipumsi_00001.dat.gz",
            "bytes": 79601908,
            "sha256": "747ff59b322fa88c067b6f1649ed1af44fd0895a55bfbb10a8a93c2960ffedf6"
        },
        "rcommandFile": {
            "url": "https://api.ipums.org/downloads/ipumsi/api/v1/extracts/1234567/ipumsi_00001.R",
            "bytes": 409,
            "sha256": "9fd213cebe2ce79be8da3a9cac6998344b6a4a769010ca8c671fd535243b19c8"
        },
        "spssCommandFile": {
            "url": "https://api.ipums.org/downloads/ipumsi/api/v1/extracts/1234567/ipumsi_00001.sps",
            "bytes": 73654,
            "sha256": "6021679b851e86a3d11b380a1a407d2ceb8b0a521e312bfac2615f49c29d7c30"
        },
        "sasCommandFile": {
            "url": "https://api.ipums.org/downloads/ipumsi/api/v1/extracts/1234567/ipumsi_00001.sas",
            "bytes": 70776,
            "sha256": "55cdfc475d61b50d5b004fc9ef0feb48e6ccc132c9cfdbcb01503d64bd548fc3"
        }
    },
    "extractDefinition": {
        "version": 2,
        "dataStructure": {
            "rectangular": {
                "on": "P"
            }
        },
        "dataFormat": "fixed_width",
        "caseSelectWho": "individuals",
        "description": "Example extract",
        "samples": {
            "mx2015a": {},
            "cl2017a": {}
        },
        "variables": {
            "COUNTRY": {
                "preselected": true
            },
            "YEAR": {
                "preselected": true
            },
            "SAMPLE": {
                "preselected": true
            },
            "SERIAL": {
                "preselected": true
            },
            "HHWT": {
                "preselected": true
            },
            "PERNUM": {
                "preselected": true
            },
            "PERWT": {
                "preselected": true
            },
            "AGE": {},
            "SEX": {},
            "EDATTAIN": {},
            "GEOMIG1_5": {}
        },
        "collection": "ipumsi"
    }
}

Retrieving Your Extract

To retrieve a completed extract, we can do so using the extract’s number.

# The IPUMS International data collection is not yet supported in the ipumspy library.

# The IPUMS International data collection is not yet supported in the ipumsr library.

# download the data file using link that came back in extract request status object once completed
curl -H "Authorization: $IPUMS_API_KEY" https://api.ipums.org/downloads/ipumsi/api/v1/extracts/1234567/ipumsi_00010.dat.gz > my_ipumsi_extract_10_dat.gz
# repeat for the other files e.g. codebook etc...

Now you are ready for further processing and analysis as you desire.

Get a Listing of Recent Extract Requests

You may also find it useful to get a historical listing of your extract requests.

# The IPUMS International data collection is not yet supported in the ipumspy library.

# The IPUMS International data collection is not yet supported in the ipumsr library.

curl -X GET \
  https://api.ipums.org/extracts?collection=ipumsi&version=beta \
  -H 'Content-Type: application/json' \
  -H "Authorization: $IPUMS_API_KEY"

# If you omit an extract number in your API call, by default this will return the 10 most recent extract requests. To adjust the amount returned, you may optionally specify a `?limit=##` parameter to get the ## most recent extracts instead.