We published “A Gentle Introduction to the X-Force Exchange API” when the IBM X-Force Exchange threat intelligence sharing portal was unveiled in April 2015, but it’s become dated in the year since. It’s time to provide some updates.
The basic ideas remain the same, but the palette of available endpoints grew, and two significant changes occurred: Anonymous access disappeared and rate limits (and commercial accounts) appeared. From the perspective of an X-Force Exchange API (XFE API) client, these differences change the authentication method and the details of responses that clients should expect.
The rest of this article examines authorized authentication and managing rate limiting. Readers should be familiar with the basic introduction to the API and treat this as an update.
API Documentation
The primary documentation for the X-Force Exchange API resides on the public documentation page. This page provides a general description of the purpose of each endpoint, as well as the details of the arguments required and the responses provided by the endpoint.
That page also describes how to acquire an API key and password. After logging in to the X-Force Exchange portal, follow the instructions to generate an API key and password. The API client needs the API key and password pair to authenticate.
The XFE API key and password pairs do not expire. However, only one valid pair can exist at a time for a single user account (IBM ID). If you generate a new API key and password, the old pair will cease to be valid. Record at least the API password somewhere because you cannot retrieve it later; you can only generate a new pair and revokes the old.
Experimenting With the Endpoints
You can experiment with the API on the public documentation page for the X-Force Exchange API. If you have an API key and password, the page allows you to enter arguments for the different API endpoints, run the request and examine the raw results of the response. Access the page, enter your API key and password in the entry boxes at the top of the page and click refresh.
In addition, the documentation page offers the details of the request arguments and the data model for the results of requests to that endpoint. It also provides an example curl command to make the request and an example JSON response. There are a couple of caveats:
- You must provide your API key and password to the page and click the refresh button. After that, you can use the controls on the page to experiment with the API endpoints.
- The displays will show you the request in the form of a curl command line.
- They will also show you the entire response as formatted JSON.
Accessing the X-Force Exchange API
X-Force Exchange API clients can use any programming language they desire as long as it can create HTTPS requests and receive HTTPS responses. The original article discussed three unaffiliated projects on Github that provide libraries to access the XFE API in three different languages:
Since the original article was published, X-Force Exchange developer Cameron Will added the IBM X-Force Exchange Node SDK project on IBM developerWorks to extend X-Force Exchange API access to Node.js clients.
These examples may provide all the capabilities a client requires. If not, they provide an excellent basis to start extending in the directions the client needs.
How the API Works
X-Force Exchange API provides a secure, RESTful, JSON-based application programming interface for the data housed in XFE. The client makes requests by composing appropriate HTTPS URLs and request headers, and the servers respond with HTTP response codes and JSON formatted data.
Watch the on-demand webinar: Level Up Your Security with Threat Intelligence
Clients access the API through the base URL. Note that only HTTPS connections are allowed. The servers will refuse connections over HTTP.
Making Requests
Note that the X-Force Exchange API fully supports international characters and clients must be prepared to deal with that. Specifically, some existing data uses non-ASCII text, and clients will receive that data in response to appropriate requests. Enumerating the collections, for example, currently produces at least one title that requires client support of non-ASCII text. You have been warned.
Headers and Authentication
The HTTP headers of API requests provide the account authentication data. They should also specify the acceptable response format. For example, the headers should include something like:
Authorization: Basic <em>$TOKEN</em>Accept: application/json
The $TOKEN variable represents the authentication data for the IBM ID. Create this string by concatenating the API key and the API token, with a colon between them.
In Python, the client could compose the authorization header’s argument string like this:
from base64 import b64encodeauthToken = b64encode(apiKey + ':' + apiPassword)
Other languages use different syntax, but it’s often similar to this example.
We remind readers familiar with the original X-Force Exchange API specification that authentication using a single Bearer token is no longer supported, and the portal user interface no longer displays a token value. All API clients must authenticate with Basic authentication and separate API key and password, as shown.
Building the Request URL
An X-Force Exchange API request identifies the desired endpoint with the first part of the URL path. Some endpoints need only a single path component, like /version, which returns the version information for the XFE. Others endpoints use several path components, such as /user/profile, which retrieves information about the authenticated user. In each case, though, a unique path prefix identifies a single endpoint.
With the prefix selecting the desired functionality, the client specifies any parameters to a request in the suffix. The version and user profile endpoints require no parameters. However, the Get IP Report endpoint (/ipr/{ip}) requires the IP address for which a report is desired. The client appends the string rendering of the IP address to the end of the URL. To make a request for an IP address report on 8.8.8.8 (a public Google DNS name server), a client needs to make a request with the URL:
https://api.xforce.ibmcloud.com/ipr/8.8.8.8
The client also needs to ensure that any parameters are encoded to make valid URLs, since some parameters might include characters that are not permitted in a valid URL. In particular, some endpoints provide full text searching of different aspects of the XFE data. These parameters often include characters not allowed in URLs. As mentioned above, the responses can also include characters that require extended support, such as UTF-8.
Receiving Responses
A client’s first indication of the results of their requests will be the HTTP response code returned from the server. If the request encounters any problems, the response code indicates the type of problem:
- 200 (Success): No worries.
- 401 (Not Authorized): The user has not logged in or provided appropriate key and password credentials.
- 402 (Payment Required): The user requested data that is only available to paying users.
- 403 (Access Denied): The user does not have the correct permissions to access the requested data.
- 404 (Artifact Not Found): The server has no data for the arguments (for example, IP or URL reputation).
- 429 (Rate Limiting): The client is making requests faster than allowed.
If the server returns standard response code 200, the request worked, and the body of the response contains the JSON-formatted response data. The client can interpret the contents of the response with the standard JSON tools and libraries for their platform.
A JSON container represents each response payload. It can hold one or more objects, depending on the endpoint the client invoked, and the results the endpoint found. For some endpoints, objects in the JSON container will themselves be containers, and the client will have to navigate the structure to examine all the responsive data. The original article has example responses with different levels of complexity and structure.
Rate Limiting and HTTP Response Headers
The HTTP responses include three headers that allow clients to conform to the rate limiting for their account:
- x-ratelimit-remaining;
- x-ratelimit-limit; and
- x-ratelimit-reset.
Each of these headers carries an integer value. The remaining header gives the number of requests the client can make within the current rate limit interval. The limit header gives the total number of requests the client can make in any rate limit interval. The reset header gives the time at which the current rate limit interval will end in UTC epoch seconds.
The easiest way to manage the request rate is to just keep making requests until the remaining header returns zero, wait a little while and try again. More ambitious clients can use the headers to implement more complex rate management schemes.
IBM Security App Exchange
A few months ago, IBM Security added the App Exchange to X-Force Exchange. Among the first apps added to the App Exchange were ones designed to provide integration between X-Force Exchange and other IBM Security products, including the IBM QRadar SIEM system.
The App Exchange currently hosts apps supporting integration with a wide variety of systems, as well as custom properties and rules for IBM Security products. Like X-Force Exchange, App Exchange is an open platform, though you must be logged in to X-Force Exchange to download or rate apps.
Conclusion
In the time since the X-Force Exchange and XFE API went public, they have garnered a lot of interest. IBM X-Force Exchange currently includes over 200 public collections informing users of everything from threat actors to malware details to X-Force Advisories. It has over 10 times as many private collections as well, allowing user teams to collaborate to defend their networks and systems.
The vulnerabilities database now holds over 100,000 publicly disclosed vulnerabilities and expands every month. In addition, the platform itself continues to grow. The project operates on an agile methodology, which provides new features and bug fixes frequently.
We hope that this article has piqued your interest and that you will find value in using and contributing to the IBM X-Force Exchange and the IBM Security App Exchange.
Sign up for a free trial of the IBM X-Force Exchange
Research Technologist, IBM Security X-Force