IPGeolocation.io CloudQuery Source Plugin


Overview

Load IP geolocation, threat intelligence, ASN ownership and abuse contact data straight into your data warehouse. The IPGeolocation.io source plugin for CloudQuery turns the IPGeolocation.io v3 API into a set of ordinary database tables, so you can join IP intelligence against your own logs, inventories and access records using plain SQL.

There is no pipeline code to write. You list the IP addresses, AS numbers or user agent strings you care about in a YAML file, run one command, and the data appears in PostgreSQL, BigQuery, Snowflake, SQLite, S3 or any other CloudQuery destination.


What is CloudQuery?

CloudQuery is an open source data movement tool. It reads from APIs through source plugins and writes to databases and object stores through destination plugins, handling schema creation, batching, retries and incremental state along the way.

The split matters in practice: the same source plugin feeds SQLite on a laptop and Snowflake in production, and switching between them is a change to one block of YAML.

This makes CloudQuery a good fit for teams that want IP intelligence sitting next to their existing data rather than behind an API call:

  • Security engineers building enrichment tables their SIEM can join against
  • Data teams who need IP attribution in the warehouse alongside product analytics
  • Compliance teams producing repeatable, auditable geographic reports
  • Network and platform teams tracking ASN ownership across their address space

What Does This Plugin Do?

It exposes five tables mapped to the IPGeolocation.io v3 endpoints.

TableAPI EndpointWhat It Returns
ipgeolocation_ip_geolocation /v3/ipgeo Country, state, city, coordinates, timezone, currency, ASN, company, network and CDN data, with optional security, abuse, accuracy, DMA code and hostname
ipgeolocation_ip_security /v3/security Threat score, VPN, proxy, residential proxy, Tor, relay, bot, spam, cloud and corporate gateway detection
ipgeolocation_abuse_contact /v3/abuse Abuse team name, emails, phone numbers and postal address for the network block
ipgeolocation_asn /v3/asn AS name, organization, RIR, allocation status, route counts, with peers, upstreams, downstreams, routes and WHOIS on request
ipgeolocation_user_agent /v3/user-agent Browser, version, device, brand, rendering engine and operating system

Nested API responses are flattened into columns, so security.vpn_provider_names becomes vpn_provider_names and you never write a JSON path in a query. Array values are stored as comma separated text, which every destination handles the same way.


Prerequisites

  • CloudQuery CLI v5 or later
  • Go 1.22.7 or later, to build the plugin binary
  • An IPGeolocation.io API key. The free Developer plan allows 1,000 requests per day and covers the geolocation table. Get one from signing up here, no card required.
  • A destination you can write to. SQLite needs nothing beyond a file path, which makes it the easiest way to try this out.

The security, abuse, ASN and user agent endpoints require a paid plan. See the pricing page for current limits.


Installation

Clone the repository and build the binary:

git clone https://github.com/IPGeolocation/cq-source-ipgeolocation
cd cq-source-ipgeolocation
go build -o cq-source-ipgeolocation

Start the plugin. It listens on localhost:7777 and stays in the foreground, so give it its own terminal window.

./cq-source-ipgeolocation serve

CloudQuery connects to that address through the grpc registry, which is how every example below is wired up.


Configuration

A CloudQuery config is one or more YAML documents. You need a source block describing the plugin and a destination block describing where rows go.

Create ipgeo.yml :

Response Preview
1kind: source
2spec:
3  name: ipgeolocation
4  registry: grpc
5  path: "localhost:7777"
6  tables: ["ipgeolocation_ip_geolocation"]
7  destinations: ["sqlite"]
8  spec:
9    api_key: "${IPGEOLOCATION_API_KEY}"
10    ips:
11      - "8.8.8.8"
12      - "1.1.1.1"
13---
14kind: destination
15spec:
16  name: sqlite
17  path: cloudquery/sqlite
18  version: "v2.9.3"
19  spec:
20    connection_string: "./ipgeo.db"

Export your key and sync:

export IPGEOLOCATION_API_KEY="your_api_key_here"
cloudquery sync ipgeo.yml

Check the result:

sqlite3 ipgeo.db "SELECT ip, country_name, city, company_name FROM ipgeolocation_ip_geolocation;"

Two rows naming the United States, Google and Cloudflare mean the key, the plugin and the destination are all working.


1. Spec Reference

FieldTypeDefaultDescription
api_key stringrequiredYour IPGeolocation.io API key
ips list of strings [] IP addresses to look up. Empty means the caller's own public IP
asns list of strings [] AS numbers for the ASN table, for example ["15169"]
user_agents list of strings [] User agent strings for the user agent table
include_security boolCrossAdds the security object to geolocation rows. Paid plans
include_abuse boolCrossAdds the abuse contact object to geolocation rows. Paid plans
include_geo_accuracy boolCrossAdds locality , accuracy_radius and confidence . Paid plans
include_dma_code boolCrossAdds dma_code , US locations only. Paid plans
include_hostname boolCrossAdds the reverse DNS hostname . Paid plans
hostname_lookup string fallback_live Resolution mode: database , live or fallback_live
asn_include list of strings [] Optional ASN modules: peers , upstreams , downstreams , routes , whois_response , or "*"
endpoint string https://api.ipgeolocation.io API base URL
timeout string 30s Per request HTTP timeout
retry_attempts int 3 Retries for 429 and 5xx responses
concurrency int 10 Tables synced in parallel
rate_limit int 15 Maximum API requests per second
user_agent string cq-source-ipgeolocation/1.0 User-Agent header sent with each request

The spec is validated before the first request, so a bad string or an unknown asn_include value fails immediately and costs no credits.


2. Keeping Your API Key Out of Git

Reference an environment variable with ${VAR_NAME} and export it at runtime. CloudQuery expands it when the config loads, and the key never touches a tracked file.


Available Tables


1. ipgeolocation_ip_geolocation

The main table. One row per entry in ips , or a single row for the caller's public IP when ips is empty. Works on the free plan.

Key columns:

ColumnTypeDescription
ip textThe IP that was looked up
continent_code textTwo letter continent code
country_code2 textISO alpha-2 country code
country_code3 textISO alpha-3 country code
country_name textCountry name
country_capital textCapital city
state_prov textState or province
state_code textISO 3166-2 subdivision code
district textDistrict or county
city textCity
zipcode textPostal code
latitude textLatitude
longitude textLongitude
is_eu booleanTrue for EU member states
geoname_id textGeoNames identifier
calling_code textInternational dialling prefix
tld textCountry code top level domain
languages textOfficial languages, comma separated
currency_code textISO 4217 currency code
currency_name textCurrency name
currency_symbol textCurrency symbol
connection_type textConnection type
route textCIDR block containing the IP
is_anycast booleanAnycast prefix
is_cdn booleanCDN address
cdn_provider_name textCDN operator
as_number textAS number, for example AS15169
asn_organization textOrganization holding the ASN
asn_type textISP, HOSTING, EDUCATION, GOVERNMENT or BUSINESS
asn_rir textARIN, RIPE, APNIC, LACNIC or AFRINIC
company_name textCompany operating the IP
company_domain textCompany domain
timezone_name textIANA timezone name
timezone_offset doubleUTC offset in seconds
timezone_is_dst booleanDST currently in effect
hostname textReverse DNS name. Paid, opt in
locality textLocality or neighbourhood. Paid, opt in
accuracy_radius textAccuracy radius in kilometres. Paid, opt in
confidence text low , medium or high . Paid, opt in
dma_code textDesignated Market Area code, US only. Paid, opt in

With include_security: true the table also carries every security column listed below, unprefixed. With include_abuse: true it carries every abuse column with an abuse_ prefix.

Example:

SELECT ip, country_name, city, company_name, timezone_name, as_number
FROM ipgeolocation_ip_geolocation
WHERE country_code2 = 'US';

2. ipgeolocation_ip_security

Threat intelligence for each IP in ips . Requires a paid plan and costs 2 credits per lookup.

ColumnTypeDescription
ip textThe IP that was looked up
threat_score integerComposite risk score from 0 to 100
is_anonymous booleanAny anonymisation detected
is_vpn booleanKnown VPN exit node
vpn_provider_names textVPN operators, comma separated
vpn_confidence_score integerVPN detection confidence
vpn_last_seen textDate last seen as a VPN exit
is_proxy booleanKnown proxy
proxy_provider_names textProxy operators, comma separated
proxy_confidence_score integerProxy detection confidence
proxy_last_seen textDate last seen as a proxy
is_residential_proxy booleanProxy on a residential connection
is_tor booleanTor exit node
is_relay booleanRelay network such as iCloud Private Relay
relay_provider_name textRelay operator
is_known_attacker booleanFlagged for attack activity
is_bot booleanAutomated traffic
bot_confidence_score integerBot detection confidence
bot_operator_name textWho runs the bot
bot_type textBot classification
is_known_good_bot booleanDeclared, well behaved crawler
bot_last_seen textDate last seen as a bot
is_spam booleanOn spam block lists
is_cloud_provider booleanCloud or hosting infrastructure
cloud_provider_name textCloud provider name
is_corporate_gateway booleanShared corporate egress point
corporate_gateway_type textGateway type
corporate_gateway_provider_name textGateway operator

Example:

SELECT ip, threat_score, is_vpn, is_proxy, is_tor, is_residential_proxy
FROM ipgeolocation_ip_security
WHERE threat_score > 50
ORDER BY threat_score DESC;

3. ipgeolocation_abuse_contact

The contact registered with the Regional Internet Registry for reporting abuse from a network block. Requires a paid plan and costs 1 credit per lookup.

ColumnTypeDescription
ip textThe IP that was looked up
route textCIDR block the contact covers
country textCountry of the registrant
name textContact or incident response team name
organization textOrganization responsible for the block
kind textContact kind, such as group or individual
address textPostal address
emails textAbuse email addresses, comma separated
phone_numbers textAbuse phone numbers, comma separated

Example:

SELECT ip, route, organization, emails, phone_numbers
FROM ipgeolocation_abuse_contact
WHERE emails != '';

4. ipgeolocation_asn

Autonomous System details, looked up by IP address, by AS number, or both. Requires a paid plan and costs 1 credit per lookup.

ColumnTypeDescription
query_ip textThe IP this row was looked up by. Empty for AS number lookups
query_asn textThe AS number this row was looked up by. Empty for IP lookups
as_number textAS number returned by the API
asn_name textShort registered name
organization textOrganization holding the ASN
country textCountry of registration
type textISP, HOSTING, EDUCATION, GOVERNMENT or BUSINESS
domain textPrimary domain
date_allocated textAllocation date
allocation_status textRegistry status
rir textRegional Internet Registry
num_of_ipv4_routes textCount of announced IPv4 prefixes
num_of_ipv6_routes textCount of announced IPv6 prefixes
peers textPeer AS numbers. Opt in
upstreams textUpstream AS numbers. Opt in
downstreams textDownstream AS numbers. Opt in
routes textAnnounced CIDR prefixes. Opt in
whois_response textRaw WHOIS record. Opt in

The last five columns stay NULL unless you name them in asn_include . That is deliberate: a tier 1 network can announce tens of thousands of prefixes and peer with thousands of other networks, and pulling all of it on every sync makes the table slow to write and awkward to query.

Response Preview
1asn_include: ["upstreams", "downstreams"]

Route counts come back as strings, so cast them when sorting:

SELECT as_number, organization, type, num_of_ipv4_routes
FROM ipgeolocation_asn
ORDER BY CAST(num_of_ipv4_routes AS INTEGER) DESC;

5. ipgeolocation_user_agent

Parses each string in user_agents . The table is skipped when the list is empty. Requires a paid plan and costs 1 credit per string.

ColumnTypeDescription
user_agent_string textThe raw string that was parsed
name textClient name, such as Chrome or Googlebot
type textClient type: browser, bot, library
version textFull version
version_major textMajor version
device_name textDevice model
device_type textDesktop, Mobile, Tablet
device_brand textManufacturer
device_cpu textCPU architecture
engine_name textRendering engine
engine_version textEngine version
os_name textOperating system
os_version textOS version
os_build textBuild identifier

Example:

SELECT name, type, device_type, os_name, os_version
FROM ipgeolocation_user_agent
ORDER BY type;

Query Examples


1. Where are these IPs?

SELECT ip, country_name, state_prov, city, latitude, longitude, timezone_name
FROM ipgeolocation_ip_geolocation;

2. Which IPs are hiding behind anonymisation?

SELECT ip, is_vpn, is_proxy, is_tor, is_relay, vpn_provider_names, threat_score
FROM ipgeolocation_ip_security
WHERE is_anonymous = true
ORDER BY threat_score DESC;

3. Which IPs belong to cloud infrastructure rather than end users?

SELECT ip, cloud_provider_name, threat_score
FROM ipgeolocation_ip_security
WHERE is_cloud_provider = true;

4. Which automation is undeclared?

Good crawlers identify themselves. This finds traffic that behaves like a bot without doing so.

SELECT ip, bot_type, bot_operator_name, bot_confidence_score, bot_last_seen
FROM ipgeolocation_ip_security
WHERE is_bot = true AND is_known_good_bot = false
ORDER BY bot_confidence_score DESC;

5. Which IPs are corporate gateways?

A gateway can front an entire company, so blocking one is much wider than blocking a single user.

SELECT ip, corporate_gateway_provider_name, corporate_gateway_type
FROM ipgeolocation_ip_security
WHERE is_corporate_gateway = true;

6. Country breakdown across everything you synced

SELECT country_name, COUNT(*) AS ip_count
FROM ipgeolocation_ip_geolocation
GROUP BY country_name
ORDER BY ip_count DESC;

7. Who provides transit to each network?

Requires asn_include: ["upstreams"] .

SELECT as_number, organization, country, upstreams
FROM ipgeolocation_asn
WHERE upstreams IS NOT NULL AND upstreams != '';

Joining Tables

Because everything lands in one database, joins are ordinary SQL and cost nothing extra at query time. The API calls happened during the sync.


1. Location plus threat data

SELECT
  g.ip,
  g.country_name,
  g.city,
  g.company_name,
  s.threat_score,
  s.is_vpn,
  s.is_proxy,
  s.is_tor
FROM ipgeolocation_ip_geolocation g
JOIN ipgeolocation_ip_security s ON g.ip = s.ip
WHERE s.threat_score > 30
ORDER BY s.threat_score DESC;

2. Everything needed to file an abuse report

SELECT
  g.ip,
  g.country_name,
  g.city,
  g.as_number,
  g.asn_organization,
  s.threat_score,
  s.is_known_attacker,
  a.route,
  a.emails
FROM ipgeolocation_ip_geolocation g
JOIN ipgeolocation_ip_security s ON g.ip = s.ip
JOIN ipgeolocation_abuse_contact a ON g.ip = a.ip
WHERE s.is_known_attacker = true;

3. Joining against your own tables

This is the part that makes a warehouse worth the trouble. Once IP intelligence is a table, it joins to anything else you already have:

SELECT
  l.user_id,
  l.login_at,
  g.country_name,
  g.city,
  s.is_vpn,
  s.threat_score
FROM auth_logins l
JOIN ipgeolocation_ip_geolocation g ON l.source_ip = g.ip
JOIN ipgeolocation_ip_security s ON l.source_ip = s.ip
WHERE s.threat_score > 40
  AND l.login_at > NOW() - INTERVAL '7 days';

Boolean literals above follow PostgreSQL. SQLite stores booleans as 0 and 1, so write is_vpn = 1 there.


Real World Use Cases


1. Building a Security Enrichment Table

Most SIEM platforms can join against a lookup table far more cheaply than they can call an external API per event. Syncing your known and suspicious IPs once a day gives the SIEM a local table to work from.

Response Preview
1kind: source
2spec:
3  name: ipgeolocation
4  registry: grpc
5  path: "localhost:7777"
6  tables:
7    - "ipgeolocation_ip_geolocation"
8    - "ipgeolocation_ip_security"
9  destinations: ["postgresql"]
10  spec:
11    api_key: "${IPGEOLOCATION_API_KEY}"
12    ips:
13      - "185.220.101.1"
14      - "2.56.188.34"
15      - "45.142.212.100"
16    include_security: true
17    rate_limit: 5

Run it on a schedule, point your detection rules at the resulting tables, and the enrichment happens at query time with no per event API cost.


2. Compliance Reporting on Geographic Access

If your service is restricted by jurisdiction, the auditable question is not "which country is this IP in today" but "which country did we record it in when access was granted". A synced table answers that, because the rows are timestamped by the sync that wrote them.

SELECT ip, country_name, country_code2, city, asn_organization
FROM ipgeolocation_ip_geolocation
WHERE country_code2 IN ('RU', 'KP', 'IR', 'SY', 'CU');

Keeping historical snapshots also lets you show what was known at the time rather than what is true now, which is usually what an auditor is asking for.


3. Vendor and Third Party Network Inventory

Platform teams often hold a list of vendor IP ranges in a spreadsheet nobody has checked in two years. Syncing them produces a table you can diff against the previous run.

SELECT query_ip, as_number, organization, type, country, rir
FROM ipgeolocation_asn
WHERE query_ip != ''
ORDER BY organization;

The type column separates hosting providers from ISPs, education networks and enterprise allocations, which is often the first thing to check when a vendor claims traffic came from their corporate office.


4. Incident Response and Abuse Reporting

When malicious traffic comes from a network you do not control, you need the right contact and the right block. One sync gives you both.

SELECT ip, organization, route, emails, phone_numbers, country
FROM ipgeolocation_abuse_contact
WHERE ip = '45.142.212.100';

The route column is the CIDR block the contact is registered for, which tells you the scope a report or a block would cover.


5. Traffic Analysis from Web Server Logs

Extract distinct user agent strings from your access logs, feed them through the user agent table, and you get a browser and bot breakdown without maintaining a parsing library.

SELECT type, name, COUNT(*) AS hits
FROM ipgeolocation_user_agent
GROUP BY type, name
ORDER BY hits DESC;

6. Timezone Aware Scheduling

timezone_name returns an IANA identifier, which most date libraries accept directly.

SELECT ip, country_name, city, timezone_name, timezone_offset
FROM ipgeolocation_ip_geolocation;

API Credits Reference

Each lookup spends credits from your plan. Multiply by the number of entries in the matching input list.

TableEndpointCredits per lookup
ipgeolocation_ip_geolocation , base fields /v3/ipgeo 1
ipgeolocation_ip_geolocation , with security and abuse /v3/ipgeo Up to 4
ipgeolocation_ip_security /v3/security 2
ipgeolocation_abuse_contact /v3/abuse 1
ipgeolocation_asn /v3/asn 1
ipgeolocation_user_agent /v3/user-agent 1

The geo accuracy, DMA code and hostname modules add columns without adding credits.

Two ways to spend more than you meant to:

  • Syncing ipgeolocation_ip_security while also setting include_security: true . That fetches the same data twice and bills for both. Pick the wide geolocation table or the dedicated security table, not both.
  • Setting tables: ["*"] on a free plan. Every paid table returns 401 for every IP, and while those failures cost no credits, the sync looks successful and the tables come back empty.

The free Developer plan allows 1,000 requests per day. Paid plans start at 150,000 requests per month. Current figures are on the pricing page.


Scheduling Syncs

cloudquery sync is one command, so any scheduler works. A cron entry for a nightly run:

0 3 * * * cd /opt/ipgeo && IPGEOLOCATION_API_KEY=xxx /usr/local/bin/cloudquery sync ipgeo.yml >> sync.log 2>&1

The plugin binary has to be serving when the sync runs. Under systemd, a long running unit for cq-source-ipgeolocation serve plus a timer for the sync keeps the two concerns separate.

Size the interval against your credits. Geolocation data for a given IP changes slowly, so daily is enough for most uses. Threat signals move faster, and hourly is reasonable for a small list of IPs you actively monitor.


Troubleshooting


1. The sync succeeds but a table is empty

Failed lookups are logged as warnings and skipped so one bad address cannot abort a run. An empty table usually means every lookup failed the same way. Re run with --log-level debug and look for failed to fetch lines, which name the IP and the error.

The usual causes: a paid table on a free key, an ips list made up entirely of private ranges, or user_agents left empty, which skips the user agent table silently.


2. Columns are NULL when you expected values

NULL means the module was never requested. Check the matching flag: include_security , include_abuse , include_geo_accuracy , include_dma_code , include_hostname , or asn_include .

An empty string means something different. It means the module ran and the API had nothing to return, which is normal for dma_code outside the US and for upstreams on a network that buys transit from nobody.

Because of this, write filters positively. WHERE is_vpn = true works whether or not the module ran, while WHERE is_vpn != true drops NULL rows in most SQL engines.


3. HTTP status codes

StatusMeaning
400Malformed request, usually an invalid IP or AS number
401Invalid key, or an endpoint your plan does not cover
403Key disabled or blocked origin
404No data for the queried resource
423Bogon address: private, loopback, reserved or multicast
429Rate limit or daily quota exhausted
5xxServer fault, retried automatically

429 and 5xx responses are retried with backoff of 1 second, then 2, then 4. Other 4xx responses are returned immediately.


4. Rate limits

rate_limit is a token bucket in requests per second. Lower it if your plan is tight or if other systems share the key. concurrency controls parallel table syncs and does not raise the request rate, since the limiter applies across the whole sync.


Building from Source

git clone https://github.com/IPGeolocation/cq-source-ipgeolocation
cd cq-source-ipgeolocation
go build ./...
go build -o cq-source-ipgeolocation

Run the tests:

go test ./... -v

The suite runs against a stub HTTP server, so it needs no API key, works offline and spends no credits. It covers spec validation, retry and rate limiting behaviour, response flattening for all five tables, and a check that every documented API field reaches a column.

The repository also ships eight example configs under example/ , covering the free plan, security focused syncs, ASN topology, user agent parsing, edge cases such as IPv6 and bogons, a PostgreSQL destination, and caller IP auto detection.


Related Resources


Frequently Asked Questions

No. The free Developer plan allows 1,000 requests per day and covers the ipgeolocation_ip_geolocation table with its base fields, which is enough for development and small syncs. The security, abuse, ASN and user agent tables need a paid plan, as do the optional modules such as include_security and include_abuse .
Steampipe queries the API live, so every SELECT makes API calls and the results are always current but never stored. CloudQuery syncs data into a database you own, so queries are free and fast afterwards but reflect the last sync. Use Steampipe for ad hoc investigation, CloudQuery for enrichment tables other systems join against.
Every CloudQuery destination, including PostgreSQL, SQLite, BigQuery, Snowflake, ClickHouse, DuckDB, S3 and GCS. Source plugins do not care where rows are written, so switching destinations is a change to one YAML block.
Yes. Put IPv6 addresses in ips exactly as you would IPv4. Coverage varies by prefix, and some IPv6 blocks return less location detail than comparable IPv4 space.
No. Resolve the name to an address first. The plugin sends whatever is in ips directly to the API.
It is logged as a warning and skipped, and the sync continues with the rest. Malformed addresses, private ranges and unsupported prefixes will not abort a run.
Leave ips out of the spec. The geolocation, security and abuse tables then query whatever public address the syncing machine egresses from, which is a quick way to check what a network looks like from outside.
NULL means the module was never requested, while false means the API checked and found nothing. Keeping the two distinct stops an unfetched IP from looking like a clean one. Set include_security: true to populate them.
Those modules are opt in through asn_include . The default keeps rows compact, because large transit networks return peer and route lists in the thousands. Add only the modules you query, or "*" for all of them.
Count the lookups: 1 credit for geolocation, 2 for security, 1 for abuse, 1 for ASN, 1 for a user agent string, multiplied by the number of entries in each input list. Enabling include_security and include_abuse raises a geolocation lookup to as many as 4 credits.
Yes. cloudquery sync is a single command, so cron, systemd timers, GitHub Actions, Airflow and similar tools all work. Keep the plugin binary serving while the sync runs, and size the interval against your daily credit allowance.
No. Every sync makes fresh API calls. To spend less, sync less often or shorten the ips list. Geolocation data changes slowly for most addresses, so daily is enough for many uses.
Go 1.22.7 or later, matching the go directive in go.mod .

Subscribe to Our Newsletter

Get the latest in geolocation tech, straight to your inbox.