Explore IPUMS NHGIS Metadata

Below we provide examples in R and curl showing how to work with our Data Extract API to explore NHGIS aggregate metadata.

Get your key from https://account.ipums.org/api_keys. Make sure to replace 'MY_KEY' (all caps) in the snippet below with your key.

R Workflow

Load Libraries and Set Key

You may have to install the httr and jsonlite libraries if they are not already installed.

library(httr)
library(jsonlite)
my_key <- MY_KEY

Get High-Level Metadata for All Datasets

result <- GET("https://api.ipums.org/des/metadata/agg/nhgis/vbeta/datasets", add_headers(Authorization = my_key))
res_df <- content(result, "parsed", simplifyDataFrame = TRUE)
res_df # Contains metadata

res_df will contain a response like:

          name                              description
1         1790_cPop                         Population Data [US, States & Counties]
2         1800_cPop                         Population Data [US, States & Counties]
3         1810_cPop                         Population Data [US, States & Counties]
4         1820_cPop                         Population Data [US, States & Counties]
5         1830_cPop                         Population Data [US, States & Counties]
6          1840_cAg                         Agriculture Data [US, States & Counties]
7        1840_cPopX                         Population & Other Data [US, States & Counties]
8         1840_cMfg                         Manufacturing Data [US, States & Counties]
9          1850_cAg                         Agriculture Data [US, States & Counties]
...

Get Detailed Metadata for a Single Dataset


    result <- GET("https://api.ipums.org/des/metadata/agg/nhgis/vbeta/datasets/1790_cPop", add_headers(Authorization = my_key))
results <- content(result, "parsed", simplifyDataFrame = TRUE)
results # Contains metadata

results will contain a response like:

    $name
[1] "1790_cPop"

$description
[1] "Population Data [US, States & Counties]"

$data_tables
   name                                                        description
1   NT1                                                   Total Population
2  NT10                                           Number of Family Members
3  NT12                                    White Population by Nationality
4  NT13                                    Families by Slaveholding Status
5  NT14                            Families by Slaveholding Status by Race
6  NT15                        White Family Members by Slaveholding Status
7  NT16 Average Number of Members in White Families by Slaveholding Status
8  NT17                 Percent of Families by Slaveholding Status by Race
9  NT18                                             Total Number of Slaves
10 NT19                   Average Number of Slaves per Slaveholding Family
11  NT2              Urban Population (Incorporated Places 2,500 and Over)
12 NT20                          Slaveholding Families by Number of Slaves
13 NT21                                                               Race
14  NT3                      Urban Population in Cities of 25,000 and Over
15  NT4              White Males and Estimated White Females by Sex by Age
16  NT5                                            White Population by Sex
17  NT6                                                  Race/Slave Status
18  NT9                                           Total Number of Families

$geog_levels
    name   description
1 county State--County
2  state         State
3 nation        Nation

Python Workflow

Get High-Level Metadata for All Datasets

To list high-level metadata for all available datasets:


import requests
import json
from pprint import pprint

my_key = MY_KEY
my_headers = {"Authorization": my_key}

nhgis_metadata = requests.get("https://api.ipums.org/des/metadata/agg/nhgis/vbeta/datasets", headers=my_headers)

pprint(nhgis_metadata.json()[0:20])

    [{'description': 'Population Data [US, States & Counties]',
      'name': '1790_cPop'},
     {'description': 'Population Data [US, States & Counties]',
      'name': '1800_cPop'},
     {'description': 'Population Data [US, States & Counties]',
      'name': '1810_cPop'},
     {'description': 'Population Data [US, States & Counties]',
      'name': '1820_cPop'},
     {'description': 'Population Data [US, States & Counties]',
      'name': '1830_cPop'},
     {'description': 'Agriculture Data [US, States & Counties]',
      'name': '1840_cAg'},
     {'description': 'Population & Other Data [US, States & Counties]',
      'name': '1840_cPopX'},
     {'description': 'Manufacturing Data [US, States & Counties]',
      'name': '1840_cMfg'},
     {'description': 'Agriculture Data [US, States & Counties]',
      'name': '1850_cAg'},
     {'description': 'Population, Agriculture & Other Data [US, States & Counties]',
      'name': '1850_cPAX'},
     {'description': 'Farm Real Estate Values [States & Counties]',
      'name': '1850_1959_cFV'},
     {'description': 'Population, Agriculture & Other Data [US & States]',
      'name': '1850_sPAX'},
     {'description': 'Agriculture Data [US, States & Counties]',
      'name': '1860_cAg'},
     {'description': 'Population, Agriculture & Other Data [US, States & Counties]',
      'name': '1860_cPAX'},
     {'description': 'Population, Agriculture & Other Data [US & States]',
      'name': '1860_sPAX'},
     {'description': 'Agriculture Data [US, States & Counties]',
      'name': '1870_cAg'},
     {'description': 'Population, Agriculture & Other Data [US, States & Counties]',
      'name': '1870_cPAX'},
     {'description': 'Religious Bodies, Occupation & Government Data [US & States]',
      'name': '1870_sROG'},
     {'description': 'Population, Housing & Other Data [US & States]',
      'name': '1870_sPHX'},
     {'description': 'Manufacturing Data [US, States & Counties]',
      'name': '1870_cMfg'}]

Get Detailed Metadata for a Single Dataset

To see the table names and other metadata for a single dataset:


nhgis_1790_cPop = requests.get("https://api.ipums.org/des/metadata/agg/nhgis/vbeta/datasets/1790_cPop", headers=my_headers)
pprint(nhgis_1790_cPop.json())

    {'data_tables': [{'description': 'Total Population', 'name': 'NT1'},
                     {'description': 'Number of Family Members', 'name': 'NT10'},
                     {'description': 'White Population by Nationality',
                      'name': 'NT12'},
                     {'description': 'Families by Slaveholding Status',
                      'name': 'NT13'},
                     {'description': 'Families by Slaveholding Status by Race',
                      'name': 'NT14'},
                     {'description': 'White Family Members by Slaveholding Status',
                      'name': 'NT15'},
                     {'description': 'Average Number of Members in White Families '
                                     'by Slaveholding Status',
                      'name': 'NT16'},
                     {'description': 'Percent of Families by Slaveholding Status '
                                     'by Race',
                      'name': 'NT17'},
                     {'description': 'Total Number of Slaves', 'name': 'NT18'},
                     {'description': 'Average Number of Slaves per Slaveholding '
                                     'Family',
                      'name': 'NT19'},
                     {'description': 'Urban Population (Incorporated Places 2,500 '
                                     'and Over)',
                      'name': 'NT2'},
                     {'description': 'Slaveholding Families by Number of Slaves',
                      'name': 'NT20'},
                     {'description': 'Race', 'name': 'NT21'},
                     {'description': 'Urban Population in Cities of 25,000 and '
                                     'Over',
                      'name': 'NT3'},
                     {'description': 'White Males and Estimated White Females by '
                                     'Sex by Age',
                      'name': 'NT4'},
                     {'description': 'White Population by Sex', 'name': 'NT5'},
                     {'description': 'Race/Slave Status', 'name': 'NT6'},
                     {'description': 'Total Number of Families', 'name': 'NT9'}],
     'description': 'Population Data [US, States & Counties]',
     'geog_levels': [{'description': 'State--County', 'name': 'county'},
                     {'description': 'State', 'name': 'state'},
                     {'description': 'Nation', 'name': 'nation'}],
     'name': '1790_cPop'}

Curl Workflow

Get High-Level Metadata for All Datasets

To list high-level metadata for all available datasets:

curl -X GET \
  https://api.ipums.org/des/metadata/agg/nhgis/vbeta/datasets \
  -H 'Content-Type: application/json' \
  -H 'Authorization: MY_KEY'

This will return a JSON response like:

[
   {
      "name" : "1790_cPop",
      "description" : "Population Data [US, States & Counties]"
   },
   {
      "name" : "1800_cPop",
      "description" : "Population Data [US, States & Counties]"
   },
   {
      "name" : "1810_cPop",
      "description" : "Population Data [US, States & Counties]"
   },
   {
      "name" : "1820_cPop",
      "description" : "Population Data [US, States & Counties]"
   },
   {
      "name" : "1830_cPop",
      "description" : "Population Data [US, States & Counties]"
   },
   {
      "description" : "Agriculture Data [US, States & Counties]",
      "name" : "1840_cAg"
   },
   {
      "name" : "1840_cPopX",
      "description" : "Population & Other Data [US, States & Counties]"
   },
   {
      "description" : "Manufacturing Data [US, States & Counties]",
      "name" : "1840_cMfg"
   }
]

Get Detailed Metadata for a Single Dataset

To see the table names and other metadata for a single dataset:

curl -X GET \
  https://api.ipums.org/des/metadata/agg/nhgis/vbeta/datasets/1790_cPop \
  -H 'Content-Type: application/json' \
  -H 'Authorization: MY_KEY'

This returns:

{
   "name" : "1790_cPop",
   "description" : "Population Data [US, States & Counties]",
   "geog_levels" : [
      {
         "name" : "county",
         "description" : "State--County"
      },
      {
         "description" : "State",
         "name" : "state"
      },
      {
         "name" : "nation",
         "description" : "Nation"
      }
   ],
   "data_tables" : [
      {
         "description" : "Total Population",
         "name" : "NT1"
      },
      {
         "description" : "Number of Family Members",
         "name" : "NT10"
      },
      {
         "name" : "NT12",
         "description" : "White Population by Nationality"
      },
      {
         "name" : "NT13",
         "description" : "Families by Slaveholding Status"
      },
      {
         "name" : "NT14",
         "description" : "Families by Slaveholding Status by Race"
      },
      {
         "description" : "White Family Members by Slaveholding Status",
         "name" : "NT15"
      },
      {
         "description" : "Average Number of Members in White Families by Slaveholding Status",
         "name" : "NT16"
      },
      {
         "description" : "Percent of Families by Slaveholding Status by Race",
         "name" : "NT17"
      },
      {
         "description" : "Total Number of Slaves",
         "name" : "NT18"
      },
      {
         "name" : "NT19",
         "description" : "Average Number of Slaves per Slaveholding Family"
      },
      {
         "description" : "Urban Population (Incorporated Places 2,500 and Over)",
         "name" : "NT2"
      },
      {
         "name" : "NT20",
         "description" : "Slaveholding Families by Number of Slaves"
      },
      {
         "description" : "Race",
         "name" : "NT21"
      },
      {
         "description" : "Urban Population in Cities of 25,000 and Over",
         "name" : "NT3"
      },
      {
         "name" : "NT4",
         "description" : "White Males and Estimated White Females by Sex by Age"
      },
      {
         "name" : "NT5",
         "description" : "White Population by Sex"
      },
      {
         "description" : "Race/Slave Status",
         "name" : "NT6"
      },
      {
         "name" : "NT9",
         "description" : "Total Number of Families"
      }
   ]
}