WINTR proxy and web scraping API documentation

An user guide to understand how does WINTR API work and how to coordinate it with your own application in order to realize an efficient and completely customized data extractor. All examples are provided in Bash, Javascript and PHP.

Getting started

Data scraping and parsing endpoint.

POST - https://api.wintr.com/fetch

Parameters:

Name
Description
Example
Options
(string) (required) url
Url to extract data from
https://api.wintr.com/test
* (default=null)
(string) (required) apikey
WINTR API key
5d6bf3bd36
* (default=null)
(array) countrycodes
Custom geolocation
["uk", "fr"]
"us"|"uk"|"de"|"nl"|"fr"|"es"|"it"|"cz"|"pl" (default=null)
(string) proxytype
The type of proxy you want to use
residential
"local"|"residential" (default="local")) - CONSUMES 15 API CREDITS (instead of 1)
(string) session
The name of your session if you want to keep the same IP across multiple requests
my_session_name
* (default=null) - ONLY WORKS IF proxytype="residential"
(string) referer
Custom referer
https://www.google.com
* (default=null)
(string) useragent
Custom user agent
Mozilla/5.0
* (default=null)
(object) headers
Custom headers
{ "X-header1": "value1", "X-header2": "value2" }
* (default=null)
(string) method
Request method
POST
"GET"|"PATCH"|"POST"|"PUT" (default="GET")
(object) body
Request input data
{ "name1": "value1", "name2": "value2" }
* (default=null)
(object) auth
HTTP authentication
{ "user": "user123", "pass": "password123" }
* (default=null)
(bool) jsrender
Javascript rendering
true
true|false (default=false) - CONSUMES 10 API CREDITS (instead of 1)
(bool) loadall
Load all assets (ads, analytics...)
true
true|false (default=false) - ONLY WORKS IF jsrender=true
(number|string) waitfor
Wait for time (in seconds) or CSS selector to appear
".navigation"
* (default=null) - ONLY WORKS IF jsrender=true
(object) outputschema
JSON output schema
see below
see below
GENERAL INFO:
  • the maximum scrapable document size is set to 3mb
  • the timeout is set to 30 seconds for each try
  • if a request is failing it will automatically retry, 3 tries maximum don't forget to set your timeout to atleast 100 seconds to ensure everything goes smoothly
  • on each try, the proxy ip address will be different and if you did not set them, some headers might be automatically set:
    • User-Agent
    • Accept
    • Accept-Encoding
    • Upgrade-Insecure-Requests
    • Sec-Fetch-User
    • Sec-Fetch-Mode
    • Accept-Language
    • Referer
PROXY INFO:
  • by default, the program is using a datacenter (local) proxy IP address to retrieve the data, you can use residential proxies for websites that are hard to scrape
  • if you want too keep the same IP address across multiple requests, you can use the session (string) parameter to proceed, your session has a duration of 120 seconds. This feature only works with residential proxy requests.
  • if you want to define the geolocation of your session, you have to set the countrycodes (array) parameter with one and only country code at the creation of the session, if you created the session before setting the countrycodes (array) parameter, you need to create a session with a different name to proceed or wait the current one to expire after 120 seconds.
JAVASCRIPT RENDERING INFO:
  • by default, when using Javascript rendering scraping, ads, videos, analytics scripts and common assets are not loaded, use the loadall (boolean) parameter to fetch everything (requests might be slowed down)
  • by default, when using Javascript rendering scraping, the program waits for the DOM to be loaded to return the data, use the waitfor (number|string) parameter with a (number) in seconds to wait for the desired amount of time or a (string) CSS selector to wait for the desired element to appear (requests might be slowed down). This feature has a timeout of 15 seconds
OUTPUT SCHEMA INFO:
  • if you want to parse the HTML content the outputschema (object) should respect certain conditions:
    • the outputschema (object) child object(s) can only contain those keys:
      • group (string): the main CSS selector
      • data (object): the data structure
    • the data (object) child object(s) must contain those keys:
      • selector (string): the child CSS selector
      • attr (string): the HTML attribute or "*html*" or "*text*"
      and optionally those keys:
      • modifier (array): a list of modification to apply to the data
      • replacer (array): a search and replace based array objects
    • if present, the modifier (array) accepted values are those (strings):
      • "touppercase": turn data to uppercase strings
      • "tolowercase": turn data to lowercase strings
      • "tonumber": turn data to primitive numbers
      • "totrimmed": turn data to trimmed strings
    • if present, the replacer (array) accepted values are (objects) which must contain 2 (strings):
      • search (string)
      • replace (string)
    • it will search and replace in the data
PARSING INFO:
  • The parsing process is made in the order you set the modifier (array) and replacer (array) elements in your outputschema (object)
  • each modification or replacement will be done in the order of the provided (arrays)
STATUS CODES & ERRORS:
  • 200 - OK => Request completed succesfully
  • 400 - BAD_REQUEST => Incorrect parameter(s)
  • 402 - PAYMENT_REQUIRED => Plan quota exceeded, please recharge your account
  • 444 - ERR_NO_RESPONSE => Target URL responded with XXX error code
  • 415 - ERR_UNSUPPORTED_MEDIA_TYPE => Binary files are not supported
  • 403 - ERR_TOO_BIG => Response size is bigger than 3MB
  • 408 - ERR_TIMEOUT => Timeout of 30 seconds exceeded
  • 404 - ERR_NOT_FOUND => Target URL unreachable
  • 5XX - UNKNOWN_ERROR => You won't be charged for this request

Saved requests endpoint

GET - https://api.wintr.com/savedrequest/:name

Parameters:

Name
Description
Example
Options
(string) (required) name
Name of the skeleton request you previously created in your dashboard query builder
getAmazonProductPrices
* (default=null)
(string) (required) apikey
WINTR API key
5d6bf3bd36
* (default=null)
(string) url
An URL that will override the saved request predefined target URL
https://api.wintr.com/test
* (default=null)
(urlencoded object string) headers
An urlencoded object of headers that will override the saved request predefined headers
{ "X-header1": "value1", "X-header2": "value2" }
* (default=null)
(urlencoded object string) body
An urlencoded object of form data that will override the saved request predefined body
{ "name1": "value1", "name2": "value2" }
* (default=null)

Account endpoint.

GET - https://api.wintr.com/accountdata

Parameters:

Name
Description
Example
Options
(string) (required) data
Data you want to retrieve
analytics
"info"|"analytics" (default="info")
(string) (required) apikey
WINTR API key
5d6bf3bd36
* (default=null)

Basic usage

Scrape a webpage.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key" }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key'
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key'
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Custom geo

Scrape a webpage with custom geolocation.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "countrycodes": ["custom_country_codes"] }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
countrycodes: [
'custom_country_codes'
]
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'countrycodes' => array(
'custom_country_codes'
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 10
},
"content": "html..."
}

Custom referer

Scrape a webpage with custom referer.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "referer": "custom_referer" }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
referer: 'custom_referer'
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'referer' => 'CUSTOM_REFERER'
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Custom useragent

Scrape a webpage with custom user agent.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "useragent": "custom_useragent" }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
useragent: 'custom_useragent'
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'useragent' => 'CUSTOM_USERAGENT'
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Custom headers

Scrape a webpage with custom headers.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "headers": { "X-header1": "value1", "X-header2": "value2" } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
headers: {
'x-header1': 'value1',
'x-header2': 'value2'
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'headers' => array(
'x-header1' => 'value1',
'x-header2' => 'value2'
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Custom method

Scrape a webpage with custom method and send data.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "method": "post", "body": { "name1": "value1", "name2": "value2" } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
method: 'POST',
body: {
'NAME1': 'VALUE1',
'NAME2': 'VALUE2'
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'method' => 'POST',
'body' => array(
'name1' => 'value1',
'name2' => 'value2'
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

HTTP authentication

Scrape a webpage protected by an HTTP authentication.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "auth": { "user": "username", "pass": "password" } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
auth: {
'user': 'username',
'pass': 'password'
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'auth' => array(
'user' => 'username',
'pass' => 'password'
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

JS rendering

Scrape a Javascript rendered webpage.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "jsrender": true }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
jsrender: true
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'jsrender' => true
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 10
},
"content": "html..."
}

Execute saved request

Execute a skeleton request previously created in your dashboard query builder and override the target url

curl 'https://api.wintr.com/savedrequest/SAVED_REQUEST_NAME?url=CUSTOM_URL&apikey=wintr_api_key'

const request = require('request-promise')

request('https://api.wintr.com/savedrequest/SAVED_REQUEST_NAME?url=CUSTOM_URL&apikey=wintr_api_key')
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/savedrequest/SAVED_REQUEST_NAME?url=CUSTOM_URL&apikey=wintr_api_key');

curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Output schema

Scrape a webpage and parse its content.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "outputschema": { "links": { "group": "body", "data": { "link_anchors": { "selector": "a", "attr": "*text*" }, "link_targets": { "selector": "a", "attr": "href" } } } } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
'outputschema': {
'links': {
'group': 'body',
'data': {
'link_anchors': {
'selector': 'a',
'attr': '*text*'
},
'link_targets': {
'selector': 'a',
'attr': 'href'
}
}
}
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'outputschema' => array(
'links' => array(
'group' => 'body',
'data' => array(
'link_anchors' => array(
'selector' => 'a',
'attr' => '*text*'
),
'link_target' => array(
'selector' => 'a',
'attr' => 'href'
)
)
)
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 5
},
"links": {
"link_anchors": [ ... ],
"link_targets": [ ... ]
}
}

Filtered output schema

Scrape a webpage, parse and filter its content.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "outputschema": { "links": { "group": "body", "data": { "link_anchors_without_linebreak_AND_trimmed_AND_to_uppercase": { "selector": "a", "attr": "*text*", "replacer": [ { "search": "\n", "replace": "" } ], "modifier": [ "totrimmed", "touppercase" ] }, "link_target": { "selector": "a", "attr": "href" } } } } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
'outputschema': {
'links': {
'group': 'body',
'data': {
'link_anchors_without_linebreak_AND_trimmed_AND_to_uppercase': {
'selector': 'a',
'attr': '*text*',
'replacer': [
{
'search': '\n',
'replace': ''
}
],
'modifier': [
'totrimmed',
'touppercase'
]
},
'link_targets': {
'selector': 'a',
'attr': 'href'
}
}
}
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'outputschema' => array(
'links' => array(
'group' => 'body',
'data' => array(
'link_anchors_without_linebreak_AND_trimmed_AND_to_uppercase' => array(
'selector' => 'a',
'attr' => '*text*',
'replacer' => array(
array(
'search' => '\n',
'replace' => ''
)
),
'modifier' => array(
'totrimmed',
'touppercase'
)
),
'link_target' => array(
'selector' => 'a',
'attr' => 'href'
)
)
)
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 5
},
"links": {
"link_anchors_without_linebreak_AND_trimmed_AND_to_uppercase": [ ... ],
"link_targets": [ ... ]
}
}

Account balance

Get account balance.

curl 'https://api.wintr.com/accountdata?data=info&apikey=wintr_api_key'

const request = require('request-promise')

request('https://api.wintr.com/accountdata/?data=info&apikey=wintr_api_key')
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/accountdata?data=info&apikey=wintr_api_key');

curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"email": "account_email",
"apicredits": account_api_credit_balance
}

Account analytics

Get account analytics (last 7 days).

curl 'https://api.wintr.com/accountdata?data=analytics&apikey=wintr_api_key'

const request = require('request-promise')

request('https://api.wintr.com/accountdata/?data=analytics&apikey=wintr_api_key')
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/accountdata?data=analytics&apikey=wintr_api_key');

curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"email": "account_email",
"analytics": [ ... ]
}