Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ python-zyte-api
:target: https://codecov.io/gh/zytedata/zyte-api
:alt: Coverage report

Python client libraries for `Zyte Data API`_.
Python client libraries for `Zyte API`_.

Command-line utility and asyncio-based library are provided by this package.

Expand All @@ -34,7 +34,7 @@ Installation
API key
=======

Make sure you have an API key for the `Zyte Data API`_ service.
Make sure you have an API key for the `Zyte API`_ service.
You can set ``ZYTE_API_KEY`` environment
variable with the key to avoid passing it around explicitly.

Expand All @@ -46,4 +46,4 @@ License is BSD 3-clause.
* Source code: https://github.com/zytedata/python-zyte-api
* Issue tracker: https://github.com/zytedata/python-zyte-api/issues

.. _Zyte Data API: https://docs.zyte.com/zyte-api/get-started.html
.. _Zyte API: https://docs.zyte.com/zyte-api/get-started.html
20 changes: 10 additions & 10 deletions docs/command_line.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Then run a script, to get the results:

.. note:: You may use ``python -m zyte_api`` instead of ``zyte-api``.

Requests to get browser HTML from those input URLs will be sent to Zyte Data
API, using up to 20 parallel connections, and the API responses will be stored
in the ``res.jsonl`` `JSON Lines`_ file, 1 response per line.
Requests to get browser HTML from those input URLs will be sent to Zyte API,
using up to 20 parallel connections, and the API responses will be stored in
the ``res.jsonl`` `JSON Lines`_ file, 1 response per line.

.. _JSON Lines: https://jsonlines.org/

Expand All @@ -34,7 +34,7 @@ the content belongs to.

If you need more flexibility, you can customize the requests by creating
a JSON Lines file with queries: a JSON object per line. You can pass any
`Zyte Data API`_ options there. For example, you could create the following
`Zyte API`_ options there. For example, you could create the following
``requests.jsonl`` file:

.. code-block:: json
Expand All @@ -46,7 +46,7 @@ a JSON Lines file with queries: a JSON object per line. You can pass any
See `API docs`_ for a description of all supported parameters.

.. _API docs: https://docs.zyte.com/zyte-api/openapi.html
.. _Zyte Data API: https://docs.zyte.com/zyte-api/get-started.html
.. _Zyte API: https://docs.zyte.com/zyte-api/get-started.html

To get results for this ``requests.jsonl`` file, run:

Expand Down Expand Up @@ -76,11 +76,11 @@ throttling errors. They are handled by CLI automatically, but they make
extraction less efficient; please tune the concurrency options to
not hit the throttling errors (HTTP 429) often.

You may be also limited by the website speed. The Zyte Data API tries not to hit
any individual website too hard, but it could be better to limit this on
a client side as well. If you're extracting data from a single website,
it could make sense to decrease the amount of parallel requests; it can ensure
higher success ratio overall.
You may be also limited by the website speed. The Zyte API tries not to hit any
individual website too hard, but it could be better to limit this on a client
side as well. If you're extracting data from a single website, it could make
sense to decrease the amount of parallel requests; it can ensure higher success
ratio overall.

If you're extracting data from multiple websites, it makes sense to spread the
load across time: if you have websites A, B and C, don't send requests in
Expand Down
4 changes: 2 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
python-zyte-api
===============

Python client libraries for `Zyte Data API`_.
Python client libraries for `Zyte API`_.

Command-line utility and asyncio-based library are provided by this package.

Expand All @@ -25,4 +25,4 @@ Command-line utility and asyncio-based library are provided by this package.
changelog
license

.. _Zyte Data API: https://docs.zyte.com/zyte-api/get-started.html
.. _Zyte API: https://docs.zyte.com/zyte-api/get-started.html
4 changes: 2 additions & 2 deletions docs/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ Installation
API key
=======

Make sure you have an API key for the `Zyte Data API`_ service.
Make sure you have an API key for the `Zyte API`_ service.
You can set ``ZYTE_API_KEY`` environment
variable with the key to avoid passing it around explicitly.

.. _Zyte Data API: https://docs.zyte.com/zyte-api/get-started.html
.. _Zyte API: https://docs.zyte.com/zyte-api/get-started.html
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ def get_version():
setup(
name='zyte-api',
version=get_version(),
description='Python interface to Zyte Data API',
description='Python interface to Zyte API',
long_description=open('README.rst').read() + "\n\n" + open('CHANGES.rst').read(),
long_description_content_type='text/x-rst',
author='Zyte Group Ltd',
Expand Down
2 changes: 1 addition & 1 deletion zyte_api/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""
Python client libraries and command line utilities for Zyte Data API
Python client libraries and command line utilities for Zyte API
"""
12 changes: 6 additions & 6 deletions zyte_api/__main__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
""" Basic command-line interface for Zyte Data APIs. """
""" Basic command-line interface for Zyte API. """

import argparse
import json
Expand Down Expand Up @@ -77,11 +77,11 @@ def read_input(input_fp, intype):


def _main(program_name='zyte-api'):
""" Process urls from input file through Zyte Data API """
""" Process urls from input file through Zyte API """
p = argparse.ArgumentParser(
prog=program_name,
description="""
Process input URLs from a file using Zyte Data API.
Process input URLs from a file using Zyte API.
""",
)
p.add_argument("input",
Expand All @@ -107,11 +107,11 @@ def _main(program_name='zyte-api'):
help="number of connections to the API server "
"(default: %(default)s)")
p.add_argument("--api-key",
help="Zyte Data API key. "
help="Zyte API key. "
"You can also set %s environment variable instead "
"of using this option." % ENV_VARIABLE)
p.add_argument("--api-url",
help="Zyte Data API endpoint (default: %(default)s)",
help="Zyte API endpoint (default: %(default)s)",
default=API_URL)
p.add_argument("--loglevel", "-L", default="INFO",
choices=["DEBUG", "INFO", "WARNING", "ERROR"],
Expand All @@ -130,7 +130,7 @@ def _main(program_name='zyte-api'):
queries = queries[:args.limit]

logger.info(f"Loaded {len(queries)} urls from {args.input.name}; shuffled: {args.shuffle}")
logger.info(f"Running Zyte Data API (connections: {args.n_conn})")
logger.info(f"Running Zyte API (connections: {args.n_conn})")

loop = asyncio.get_event_loop()
coro = run(queries,
Expand Down
2 changes: 1 addition & 1 deletion zyte_api/aio/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""
Asyncio client for Zyte Data API
Asyncio client for Zyte API
"""
4 changes: 2 additions & 2 deletions zyte_api/aio/client.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
Asyncio client for Zyte Data API
Asyncio client for Zyte API
"""

import asyncio
Expand Down Expand Up @@ -127,7 +127,7 @@ def request_parallel_as_completed(self,
endpoint: str = 'extract',
session: Optional[aiohttp.ClientSession] = None,
) -> Iterator[asyncio.Future]:
""" Send multiple requests to Zyte Data API in parallel.
""" Send multiple requests to Zyte API in parallel.
Return an `asyncio.as_completed` iterator.

``queries`` is a list of requests to process (dicts).
Expand Down
3 changes: 1 addition & 2 deletions zyte_api/aio/retry.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# -*- coding: utf-8 -*-
"""
Zyte Data Extraction retrying logic.
Zyte API retrying logic.

TODO: add sync support; only aio is supported at the moment.
TODO: Implement retry logic for temparary errors (520) using the proposed retry-after header.
"""
import asyncio
Expand Down
2 changes: 1 addition & 1 deletion zyte_api/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

@attr.s(auto_attribs=True)
class ParsedError:
""" Parsed error from Zyte Data API """
""" Parsed error from Zyte API """
response_body: bytes
data: Optional[dict]
parse_error: Optional[str]
Expand Down