WINTR proxy and web scraping API documentation

An user guide to understand how is WINTR API working and how to coordinate it with your own application in order to realize an efficient and completely customized data extraction. All examples are provided in Bash, Javascript and PHP.

Getting started

Data scraping and parsing endpoint.

POST - https://api.wintr.com/fetch

Parameters:

Name
Description
Example
Options
(string) (required) url
Url to extract data from
https://api.wintr.com/test
* (default=null)
(string) (required) apikey
WINTR API key
5d6bf3bd36
* (default=null)
(string) countrycode
Custom geolocation
uk
"us"|"uk"|"de"|"nl"|"fr"|"es"|"it"|"cz"|"pl" (default=null)
(string) referer
Custom referer
https://www.google.com
* (default=null)
(string) useragent
Custom user agent
Mozilla/5.0
* (default=null)
(object) headers
Custom headers
{ "X-header1": "value1", "X-header2": "value2" }
* (default=null)
(string) method
Request method
POST
"GET"|"PATCH"|"POST"|"PUT" (default="GET")
(object) body
Request input data
{ "name1": "value1", "name2": "value2" }
* (default=null)
(object) auth
HTTP authentication
{ "user": "user123", "pass": "password123" }
* (default=null)
(bool) jsrender
Javascript rendering
true
true|false (default=false)
(object) outputschema
JSON output schema
{ "links_href": { "selector": "a", "attr": "href" } }
{ "*": { "selector": "*", "attr": "*html*"|"*text*"|*, "modifier": *, "replacer": *, "contains": *, "doesnotcontain": *, "removeemptystrings": * } } (default=null)
SCRAPING INFO:
  • the maximum scrapable document size is set to 6mb
  • the timeout is set to 20 seconds for each try
  • if a request is failing it will automatically retry, 3 tries maximum
  • on each try, the proxy ip address will be different and if you did not set them, some headers might be automatically set:
    • "User-Agent"
    • "Accept"
    • "Upgrade-Insecure-Requests"
    • "Sec-Fetch-User"
    • "Sec-Fetch-Mode"
    • "Accept-Language"
    • "Referer"
OUTPUT SCHEMA INFO:
  • if you want to parse the HTML content the outputschema parameter should respect certain conditions:
    • the outputschema sub-objects can only contain those parameters:
      • "selector"
      • "attr"
      • "modifier"
      • "replacer"
      • "contains"
      • "doesnotcontain"
      • "removeemptystrings"
    • the selector and attr parameters are mandatory and must be (string), the selector parameter can be any CSS selector and the attr parameter can be any HTML attribute or "*text*" or "*html*"
    • it will select the element(s) to analyze and the HTML attribute value or the text value or the HTML content to retrieve
    • the modifier parameter is optional and must be an (array) of (string), the accepted values are:
      • "touppercase"
      • "tolowercase"
      • "tonumber"
      • "totrimmed"
    • it will modify the values to turn them to uppercase strings, lowercase strings, primitive numbers and/or trimmed strings
    • replacer parameter is optional and must be an (array) of (object) which must contains 2 keys that must be (string):
      • "search"
      • "replace"
    • it will search and replace in the selected values
    • contains parameter is optional and must be an (array) of (string)
    • it will ensure that you will only retrieve the values that contains XXX, YYY, ZZZ...
    • doesnotcontain parameter is optional and must be an (array) of (string)
    • it will ensure that you will only retrieve the values that does not contain XXX, YYY, ZZZ...
    • removeemptystrings parameter is optional and must be a (bool)
    • it will remove empty values
PARSING INFO:
  • The parsing process is made in the order you set the modifier, replacer, contains, doesnotcontain & removeemptystrings elements in your outputschema
  • each modification will be done in the order of the provided (array)

Account endpoint.

GET - https://api.wintr.com/accountdata

Parameters:

Name
Description
Example
Options
(string) (required) data
Data you want to retrieve
analytics
"info"|"analytics" (default="info")
(string) (required) apikey
WINTR API key
5d6bf3bd36
* (default=null)

Basic usage

Scrape a webpage.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key" }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key'
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key'
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Custom geo

Scrape a webpage with custom geolocation.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "countrycode": "custom_country_code" }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
countrycode: 'custom_country_code'
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'countrycode' => 'CUSTOM_COUNTRY_CODE'
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 10
},
"content": "html..."
}

Custom referer

Scrape a webpage with custom referer.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "referer": "custom_referer" }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
referer: 'custom_referer'
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'referer' => 'CUSTOM_REFERER'
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Custom useragent

Scrape a webpage with custom useragent.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "useragent": "custom_useragent" }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
useragent: 'custom_useragent'
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'useragent' => 'CUSTOM_USERAGENT'
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Custom headers

Scrape a webpage with custom headers.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "headers": { "X-header1": "value1", "X-header2": "value2" } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
headers: {
'x-header1': 'value1',
'x-header2': 'value2'
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'headers' => array(
'x-header1' => 'value1',
'x-header2' => 'value2'
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

Custom method

Scrape a webpage with custom method and send data.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "method": "post", "body": { "name1": "value1", "name2": "value2" } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
method: 'POST',
body: {
'NAME1': 'VALUE1',
'NAME2': 'VALUE2'
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'method' => 'POST',
'body' => array(
'name1' => 'value1',
'name2' => 'value2'
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

HTTP authentication

Scrape a webpage protected by an HTTP authentication.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "auth": { "user": "username", "pass": "password" } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
auth: {
'user': 'username',
'pass': 'password'
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'auth' => array(
'user' => 'username',
'pass' => 'password'
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 1
},
"content": "html..."
}

JS rendering

Scrape a Javascript rendered webpage.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "jsrender": true }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
jsrender: true
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'jsrender' => true
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 10
},
"content": "html..."
}

Output schema

Scrape a webpage and parse it's content.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "outputschema": { "links_anchors": { "selector": "a", "attr": "*text*" } } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
'outputschema': {
'links_anchors': {
'selector': 'a',
'attr': '*text*'
}
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'outputschema' => array(
'links_anchors' => array(
'selector' => 'a',
'attr' => '*text*'
)
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 5
},
"content": {
"links_anchors": [ ... ]
}
}

Filtered output schema

Scrape a webpage, parse and filter it's content.

curl -X POST 'https://api.wintr.com/fetch' \
-H 'Content-Type: application/json' \
-d '{ "url": "target_url", "apikey": "wintr_api_key", "outputschema": { "https_links_with_targetblank_hrefs_in_lowercase": { "selector": "a[target=\"_blank\"]", "attr": "href", "modifier": ["tolowercase"], "contains": ["https"] } } }'

const request = require('request-promise')
const options = {
method: 'POST',
json: true,
body: {
url: 'target_url',
apikey: 'wintr_api_key',
outputschema: {
'https_links_with_targetblank_hrefs_in_lowercase': {

'selector': 'a[target="_blank"]',
'attr': 'href',
'modifier': ['tolowercase'],
'contains': ['https']
}
}
},
url: 'https://api.wintr.com/fetch'
}

request(options)
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/fetch');
$body = json_encode(array(
'url' => 'target_url',
'apikey' => 'wintr_api_key',
'outputschema' => array(
'https_links_with_targetblank_hrefs_in_lowercase' => array(
'selector' => 'a[target="_blank"]',
'attr' => 'href',
'modifier' => array('tolowercase'),
'contains' => array('https')
)
)
));

curl_setopt($request, CURLOPT_POSTFIELDS, $body);
curl_setopt($request, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"info": {
"status": 200,
"method": "GET",
"request_headers": { ... },
"response_headers": { ... },
"charged_apicredits": 5
},
"content": {
"https_links_with_targetblank_hrefs_in_lowercase": [ ... ]
}
}

Account balance

Get account balance.

curl 'https://api.wintr.com/accountdata?data=info&apikey=wintr_api_key'

const request = require('request-promise')

request('https://api.wintr.com/accountdata/?data=info&apikey=wintr_api_key')
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/accountdata?data=info&apikey=wintr_api_key');

curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"email": "account_email",
"apicredits": account_api_credit_balance
}

Account analytics

Get account analytics (last 7 days).

curl 'https://api.wintr.com/accountdata?data=analytics&apikey=wintr_api_key'

const request = require('request-promise')

request('https://api.wintr.com/accountdata/?data=analytics&apikey=wintr_api_key')
.then((result) => {
console.log(result)
})

$request = curl_init('https://api.wintr.com/accountdata?data=analytics&apikey=wintr_api_key');

curl_setopt($request, CURLOPT_RETURNTRANSFER, true);
curl_setopt($request, CURLOPT_SSL_VERIFYPEER, false);

var_dump(curl_exec($request));
curl_close($request);

Result:

{
"email": "account_email",
"analytics": [ ... ]
}