Wala shell

Advanced PHP File Manager

Server IP: 127.0.0.1 | PHP Version: 8.3.30 | Domains Hosted: 54

Name	Size	Last Modified	Actions
📁 ..	-	2024-05-29 15:36
📄 advanced.rst .. _advanced: Advanced Usage ============== This document covers some of Requests more advanced features. .. _session-objects: Session Objects --------------- The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance, and will use ``urllib3``'s `connection pooling`_. So if you're making several requests to the same host, the underlying TCP connection will be reused, which can result in a significant performance increase (see `HTTP persistent connection`_). A Session object has all the methods of the main Requests API. Let's persist some cookies across requests:: s = requests.Session() s.get('https://httpbin.org/cookies/set/sessioncookie/123456789') r = s.get('https://httpbin.org/cookies') print(r.text) # '{"cookies": {"sessioncookie": "123456789"}}' Sessions can also be used to provide default data to the request methods. This is done by providing data to the properties on a Session object:: s = requests.Session() s.auth = ('user', 'pass') s.headers.update({'x-test': 'true'}) # both 'x-test' and 'x-test2' are sent s.get('https://httpbin.org/headers', headers={'x-test2': 'true'}) Any dictionaries that you pass to a request method will be merged with the session-level values that are set. The method-level parameters override session parameters. Note, however, that method-level parameters will not be persisted across requests, even if using a session. This example will only send the cookies with the first request, but not the second:: s = requests.Session() r = s.get('https://httpbin.org/cookies', cookies={'from-my': 'browser'}) print(r.text) # '{"cookies": {"from-my": "browser"}}' r = s.get('https://httpbin.org/cookies') print(r.text) # '{"cookies": {}}' If you want to manually add cookies to your session, use the :ref:`Cookie utility functions <api-cookies>` to manipulate :attr:`Session.cookies <requests.Session.cookies>`. Sessions can also be used as context managers:: with requests.Session() as s: s.get('https://httpbin.org/cookies/set/sessioncookie/123456789') This will make sure the session is closed as soon as the ``with`` block is exited, even if unhandled exceptions occurred. .. admonition:: Remove a Value From a Dict Parameter Sometimes you'll want to omit session-level keys from a dict parameter. To do this, you simply set that key's value to ``None`` in the method-level parameter. It will automatically be omitted. All values that are contained within a session are directly available to you. See the :ref:`Session API Docs <sessionapi>` to learn more. .. _request-and-response-objects: Request and Response Objects ---------------------------- Whenever a call is made to ``requests.get()`` and friends, you are doing two major things. First, you are constructing a ``Request`` object which will be sent off to a server to request or query some resource. Second, a ``Response`` object is generated once Requests gets a response back from the server. The ``Response`` object contains all of the information returned by the server and also contains the ``Request`` object you created originally. Here is a simple request to get some very important information from Wikipedia's servers:: >>> r = requests.get('https://en.wikipedia.org/wiki/Monty_Python') If we want to access the headers the server sent back to us, we do this:: >>> r.headers {'content-length': '56170', 'x-content-type-options': 'nosniff', 'x-cache': 'HIT from cp1006.eqiad.wmnet, MISS from cp1010.eqiad.wmnet', 'content-encoding': 'gzip', 'age': '3080', 'content-language': 'en', 'vary': 'Accept-Encoding,Cookie', 'server': 'Apache', 'last-modified': 'Wed, 13 Jun 2012 01:33:50 GMT', 'connection': 'close', 'cache-control': 'private, s-maxage=0, max-age=0, must-revalidate', 'date': 'Thu, 14 Jun 2012 12:59:39 GMT', 'content-type': 'text/html; charset=UTF-8', 'x-cache-lookup': 'HIT from cp1006.eqiad.wmnet:3128, MISS from cp1010.eqiad.wmnet:80'} However, if we want to get the headers we sent the server, we simply access the request, and then the request's headers:: >>> r.request.headers {'Accept-Encoding': 'identity, deflate, compress, gzip', 'Accept': '/', 'User-Agent': 'python-requests/1.2.0'} .. _prepared-requests: Prepared Requests ----------------- Whenever you receive a :class:`Response <requests.Response>` object from an API call or a Session call, the ``request`` attribute is actually the ``PreparedRequest`` that was used. In some cases you may wish to do some extra work to the body or headers (or anything else really) before sending a request. The simple recipe for this is the following:: from requests import Request, Session s = Session() req = Request('POST', url, data=data, headers=headers) prepped = req.prepare() # do something with prepped.body prepped.body = 'No, I want exactly this as the body.' # do something with prepped.headers del prepped.headers['Content-Type'] resp = s.send(prepped, stream=stream, verify=verify, proxies=proxies, cert=cert, timeout=timeout ) print(resp.status_code) Since you are not doing anything special with the ``Request`` object, you prepare it immediately and modify the ``PreparedRequest`` object. You then send that with the other parameters you would have sent to ``requests.`` or ``Session.``. However, the above code will lose some of the advantages of having a Requests :class:`Session <requests.Session>` object. In particular, :class:`Session <requests.Session>`-level state such as cookies will not get applied to your request. To get a :class:`PreparedRequest <requests.PreparedRequest>` with that state applied, replace the call to :meth:`Request.prepare() <requests.Request.prepare>` with a call to :meth:`Session.prepare_request() <requests.Session.prepare_request>`, like this:: from requests import Request, Session s = Session() req = Request('GET', url, data=data, headers=headers) prepped = s.prepare_request(req) # do something with prepped.body prepped.body = 'Seriously, send exactly these bytes.' # do something with prepped.headers prepped.headers['Keep-Dead'] = 'parrot' resp = s.send(prepped, stream=stream, verify=verify, proxies=proxies, cert=cert, timeout=timeout ) print(resp.status_code) When you are using the prepared request flow, keep in mind that it does not take into account the environment. This can cause problems if you are using environment variables to change the behaviour of requests. For example: Self-signed SSL certificates specified in ``REQUESTS_CA_BUNDLE`` will not be taken into account. As a result an ``SSL: CERTIFICATE_VERIFY_FAILED`` is thrown. You can get around this behaviour by explicitly merging the environment settings into your session:: from requests import Request, Session s = Session() req = Request('GET', url) prepped = s.prepare_request(req) # Merge environment settings into session settings = s.merge_environment_settings(prepped.url, {}, None, None, None) resp = s.send(prepped, *settings) print(resp.status_code) .. _verification: SSL Cert Verification --------------------- Requests verifies SSL certificates for HTTPS requests, just like a web browser. By default, SSL verification is enabled, and Requests will throw a SSLError if it's unable to verify the certificate:: >>> requests.get('https://requestb.in') requests.exceptions.SSLError: hostname 'requestb.in' doesn't match either of '.herokuapp.com', 'herokuapp.com' I don't have SSL setup on this domain, so it throws an exception. Excellent. GitHub does though:: >>> requests.get('https://github.com') <Response [200]> You can pass ``verify`` the path to a CA_BUNDLE file or directory with certificates of trusted CAs:: >>> requests.get('https://github.com', verify='/path/to/certfile') or persistent:: s = requests.Session() s.verify = '/path/to/certfile' .. note:: If ``verify`` is set to a path to a directory, the directory must have been processed using the ``c_rehash`` utility supplied with OpenSSL. This list of trusted CAs can also be specified through the ``REQUESTS_CA_BUNDLE`` environment variable. If ``REQUESTS_CA_BUNDLE`` is not set, ``CURL_CA_BUNDLE`` will be used as fallback. Requests can also ignore verifying the SSL certificate if you set ``verify`` to False:: >>> requests.get('https://kennethreitz.org', verify=False) <Response [200]> Note that when ``verify`` is set to ``False``, requests will accept any TLS certificate presented by the server, and will ignore hostname mismatches and/or expired certificates, which will make your application vulnerable to man-in-the-middle (MitM) attacks. Setting verify to ``False`` may be useful during local development or testing. By default, ``verify`` is set to True. Option ``verify`` only applies to host certs. Client Side Certificates ------------------------ You can also specify a local cert to use as client side certificate, as a single file (containing the private key and the certificate) or as a tuple of both files' paths:: >>> requests.get('https://kennethreitz.org', cert=('/path/client.cert', '/path/client.key')) <Response [200]> or persistent:: s = requests.Session() s.cert = '/path/client.cert' If you specify a wrong path or an invalid cert, you'll get a SSLError:: >>> requests.get('https://kennethreitz.org', cert='/wrong_path/client.pem') SSLError: [Errno 336265225] _ssl.c:347: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib .. warning:: The private key to your local certificate must be unencrypted. Currently, Requests does not support using encrypted keys. .. _ca-certificates: CA Certificates --------------- Requests uses certificates from the package `certifi`_. This allows for users to update their trusted certificates without changing the version of Requests. Before version 2.16, Requests bundled a set of root CAs that it trusted, sourced from the `Mozilla trust store`_. The certificates were only updated once for each Requests version. When ``certifi`` was not installed, this led to extremely out-of-date certificate bundles when using significantly older versions of Requests. For the sake of security we recommend upgrading certifi frequently! .. _HTTP persistent connection: https://en.wikipedia.org/wiki/HTTP_persistent_connection .. _connection pooling: https://urllib3.readthedocs.io/en/latest/reference/urllib3.connectionpool.html .. _certifi: https://certifiio.readthedocs.io/ .. _Mozilla trust store: https://hg.mozilla.org/mozilla-central/raw-file/tip/security/nss/lib/ckfw/builtins/certdata.txt .. _body-content-workflow: Body Content Workflow --------------------- By default, when you make a request, the body of the response is downloaded immediately. You can override this behaviour and defer downloading the response body until you access the :attr:`Response.content <requests.Response.content>` attribute with the ``stream`` parameter:: tarball_url = 'https://github.com/psf/requests/tarball/main' r = requests.get(tarball_url, stream=True) At this point only the response headers have been downloaded and the connection remains open, hence allowing us to make content retrieval conditional:: if int(r.headers['content-length']) < TOO_LONG: content = r.content ... You can further control the workflow by use of the :meth:`Response.iter_content() <requests.Response.iter_content>` and :meth:`Response.iter_lines() <requests.Response.iter_lines>` methods. Alternatively, you can read the undecoded body from the underlying urllib3 :class:`urllib3.HTTPResponse <urllib3.response.HTTPResponse>` at :attr:`Response.raw <requests.Response.raw>`. If you set ``stream`` to ``True`` when making a request, Requests cannot release the connection back to the pool unless you consume all the data or call :meth:`Response.close <requests.Response.close>`. This can lead to inefficiency with connections. If you find yourself partially reading request bodies (or not reading them at all) while using ``stream=True``, you should make the request within a ``with`` statement to ensure it's always closed:: with requests.get('https://httpbin.org/get', stream=True) as r: # Do things with the response here. .. _keep-alive: Keep-Alive ---------- Excellent news — thanks to urllib3, keep-alive is 100% automatic within a session! Any requests that you make within a session will automatically reuse the appropriate connection! Note that connections are only released back to the pool for reuse once all body data has been read; be sure to either set ``stream`` to ``False`` or read the ``content`` property of the ``Response`` object. .. _streaming-uploads: Streaming Uploads ----------------- Requests supports streaming uploads, which allow you to send large streams or files without reading them into memory. To stream and upload, simply provide a file-like object for your body:: with open('massive-body', 'rb') as f: requests.post('http://some.url/streamed', data=f) .. warning:: It is strongly recommended that you open files in :ref:`binary mode <tut-files>`. This is because Requests may attempt to provide the ``Content-Length`` header for you, and if it does this value will be set to the number of bytes in the file. Errors may occur if you open the file in text mode. .. _chunk-encoding: Chunk-Encoded Requests ---------------------- Requests also supports Chunked transfer encoding for outgoing and incoming requests. To send a chunk-encoded request, simply provide a generator (or any iterator without a length) for your body:: def gen(): yield 'hi' yield 'there' requests.post('http://some.url/chunked', data=gen()) For chunked encoded responses, it's best to iterate over the data using :meth:`Response.iter_content() <requests.Response.iter_content>`. In an ideal situation you'll have set ``stream=True`` on the request, in which case you can iterate chunk-by-chunk by calling ``iter_content`` with a ``chunk_size`` parameter of ``None``. If you want to set a maximum size of the chunk, you can set a ``chunk_size`` parameter to any integer. .. _multipart: POST Multiple Multipart-Encoded Files ------------------------------------- You can send multiple files in one request. For example, suppose you want to upload image files to an HTML form with a multiple file field 'images':: <input type="file" name="images" multiple="true" required="true"/> To do that, just set files to a list of tuples of ``(form_field_name, file_info)``:: >>> url = 'https://httpbin.org/post' >>> multiple_files = [ ... ('images', ('foo.png', open('foo.png', 'rb'), 'image/png')), ... ('images', ('bar.png', open('bar.png', 'rb'), 'image/png'))] >>> r = requests.post(url, files=multiple_files) >>> r.text { ... 'files': {'images': 'data:image/png;base64,iVBORw ....'} 'Content-Type': 'multipart/form-data; boundary=3131623adb2043caaeb5538cc7aa0b3a', ... } .. warning:: It is strongly recommended that you open files in :ref:`binary mode <tut-files>`. This is because Requests may attempt to provide the ``Content-Length`` header for you, and if it does this value will be set to the number of bytes in the file. Errors may occur if you open the file in text mode. .. _event-hooks: Event Hooks ----------- Requests has a hook system that you can use to manipulate portions of the request process, or signal event handling. Available hooks: ``response``: The response generated from a Request. You can assign a hook function on a per-request basis by passing a ``{hook_name: callback_function}`` dictionary to the ``hooks`` request parameter:: hooks={'response': print_url} That ``callback_function`` will receive a chunk of data as its first argument. :: def print_url(r, args, kwargs): print(r.url) Your callback function must handle its own exceptions. Any unhandled exception won't be passed silently and thus should be handled by the code calling Requests. If the callback function returns a value, it is assumed that it is to replace the data that was passed in. If the function doesn't return anything, nothing else is affected. :: def record_hook(r, args, kwargs): r.hook_called = True return r Let's print some request method arguments at runtime:: >>> requests.get('https://httpbin.org/', hooks={'response': print_url}) https://httpbin.org/ <Response [200]> You can add multiple hooks to a single request. Let's call two hooks at once:: >>> r = requests.get('https://httpbin.org/', hooks={'response': [print_url, record_hook]}) >>> r.hook_called True You can also add hooks to a ``Session`` instance. Any hooks you add will then be called on every request made to the session. For example:: >>> s = requests.Session() >>> s.hooks['response'].append(print_url) >>> s.get('https://httpbin.org/') https://httpbin.org/ <Response [200]> A ``Session`` can have multiple hooks, which will be called in the order they are added. .. _custom-auth: Custom Authentication --------------------- Requests allows you to specify your own authentication mechanism. Any callable which is passed as the ``auth`` argument to a request method will have the opportunity to modify the request before it is dispatched. Authentication implementations are subclasses of :class:`AuthBase <requests.auth.AuthBase>`, and are easy to define. Requests provides two common authentication scheme implementations in ``requests.auth``: :class:`HTTPBasicAuth <requests.auth.HTTPBasicAuth>` and :class:`HTTPDigestAuth <requests.auth.HTTPDigestAuth>`. Let's pretend that we have a web service that will only respond if the ``X-Pizza`` header is set to a password value. Unlikely, but just go with it. :: from requests.auth import AuthBase class PizzaAuth(AuthBase): """Attaches HTTP Pizza Authentication to the given Request object.""" def __init__(self, username): # setup any auth-related data here self.username = username def __call__(self, r): # modify and return the request r.headers['X-Pizza'] = self.username return r Then, we can make a request using our Pizza Auth:: >>> requests.get('http://pizzabin.org/admin', auth=PizzaAuth('kenneth')) <Response [200]> .. _streaming-requests: Streaming Requests ------------------ With :meth:`Response.iter_lines() <requests.Response.iter_lines>` you can easily iterate over streaming APIs such as the `Twitter Streaming API <https://dev.twitter.com/streaming/overview>`_. Simply set ``stream`` to ``True`` and iterate over the response with :meth:`~requests.Response.iter_lines()`:: import json import requests r = requests.get('https://httpbin.org/stream/20', stream=True) for line in r.iter_lines(): # filter out keep-alive new lines if line: decoded_line = line.decode('utf-8') print(json.loads(decoded_line)) When using `decode_unicode=True` with :meth:`Response.iter_lines() <requests.Response.iter_lines>` or :meth:`Response.iter_content() <requests.Response.iter_content>`, you'll want to provide a fallback encoding in the event the server doesn't provide one:: r = requests.get('https://httpbin.org/stream/20', stream=True) if r.encoding is None: r.encoding = 'utf-8' for line in r.iter_lines(decode_unicode=True): if line: print(json.loads(line)) .. warning:: :meth:`~requests.Response.iter_lines()` is not reentrant safe. Calling this method multiple times causes some of the received data being lost. In case you need to call it from multiple places, use the resulting iterator object instead:: lines = r.iter_lines() # Save the first line for later or just skip it first_line = next(lines) for line in lines: print(line) .. _proxies: Proxies ------- If you need to use a proxy, you can configure individual requests with the ``proxies`` argument to any request method:: import requests proxies = { 'http': 'http://10.10.1.10:3128', 'https': 'http://10.10.1.10:1080', } requests.get('http://example.org', proxies=proxies) Alternatively you can configure it once for an entire :class:`Session <requests.Session>`:: import requests proxies = { 'http': 'http://10.10.1.10:3128', 'https': 'http://10.10.1.10:1080', } session = requests.Session() session.proxies.update(proxies) session.get('http://example.org') .. warning:: Setting ``session.proxies`` may behave differently than expected. Values provided will be overwritten by environmental proxies (those returned by `urllib.request.getproxies <https://docs.python.org/3/library/urllib.request.html#urllib.request.getproxies>`_). To ensure the use of proxies in the presence of environmental proxies, explicitly specify the ``proxies`` argument on all individual requests as initially explained above. See `#2018 <https://github.com/psf/requests/issues/2018>`_ for details. When the proxies configuration is not overridden per request as shown above, Requests relies on the proxy configuration defined by standard environment variables ``http_proxy``, ``https_proxy``, ``no_proxy``, and ``all_proxy``. Uppercase variants of these variables are also supported. You can therefore set them to configure Requests (only set the ones relevant to your needs):: $ export HTTP_PROXY="http://10.10.1.10:3128" $ export HTTPS_PROXY="http://10.10.1.10:1080" $ export ALL_PROXY="socks5://10.10.1.10:3434" $ python >>> import requests >>> requests.get('http://example.org') To use HTTP Basic Auth with your proxy, use the `http://user:password@host/` syntax in any of the above configuration entries:: $ export HTTPS_PROXY="http://user:pass@10.10.1.10:1080" $ python >>> proxies = {'http': 'http://user:pass@10.10.1.10:3128/'} .. warning:: Storing sensitive username and password information in an environment variable or a version-controlled file is a security risk and is highly discouraged. To give a proxy for a specific scheme and host, use the `scheme://hostname` form for the key. This will match for any request to the given scheme and exact hostname. :: proxies = {'http://10.20.1.128': 'http://10.10.1.10:5323'} Note that proxy URLs must include the scheme. Finally, note that using a proxy for https connections typically requires your local machine to trust the proxy's root certificate. By default the list of certificates trusted by Requests can be found with:: from requests.utils import DEFAULT_CA_BUNDLE_PATH print(DEFAULT_CA_BUNDLE_PATH) You override this default certificate bundle by setting the ``REQUESTS_CA_BUNDLE`` (or ``CURL_CA_BUNDLE``) environment variable to another file path:: $ export REQUESTS_CA_BUNDLE="/usr/local/myproxy_info/cacert.pem" $ export https_proxy="http://10.10.1.10:1080" $ python >>> import requests >>> requests.get('https://example.org') .. _socks: SOCKS ^^^^^ .. versionadded:: 2.10.0 In addition to basic HTTP proxies, Requests also supports proxies using the SOCKS protocol. This is an optional feature that requires that additional third-party libraries be installed before use. You can get the dependencies for this feature from ``pip``: .. code-block:: bash $ python -m pip install requests[socks] Once you've installed those dependencies, using a SOCKS proxy is just as easy as using a HTTP one:: proxies = { 'http': 'socks5://user:pass@host:port', 'https': 'socks5://user:pass@host:port' } Using the scheme ``socks5`` causes the DNS resolution to happen on the client, rather than on the proxy server. This is in line with curl, which uses the scheme to decide whether to do the DNS resolution on the client or proxy. If you want to resolve the domains on the proxy server, use ``socks5h`` as the scheme. .. _compliance: Compliance ---------- Requests is intended to be compliant with all relevant specifications and RFCs where that compliance will not cause difficulties for users. This attention to the specification can lead to some behaviour that may seem unusual to those not familiar with the relevant specification. Encodings ^^^^^^^^^ When you receive a response, Requests makes a guess at the encoding to use for decoding the response when you access the :attr:`Response.text <requests.Response.text>` attribute. Requests will first check for an encoding in the HTTP header, and if none is present, will use `charset_normalizer <https://pypi.org/project/charset_normalizer/>`_ or `chardet <https://github.com/chardet/chardet>`_ to attempt to guess the encoding. If ``chardet`` is installed, ``requests`` uses it, however for python3 ``chardet`` is no longer a mandatory dependency. The ``chardet`` library is an LGPL-licenced dependency and some users of requests cannot depend on mandatory LGPL-licensed dependencies. When you install ``requests`` without specifying ``[use_chardet_on_py3]`` extra, and ``chardet`` is not already installed, ``requests`` uses ``charset-normalizer`` (MIT-licensed) to guess the encoding. The only time Requests will not guess the encoding is if no explicit charset is present in the HTTP headers and** the ``Content-Type`` header contains ``text``. In this situation, `RFC 2616 <https://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.7.1>`_ specifies that the default charset must be ``ISO-8859-1``. Requests follows the specification in this case. If you require a different encoding, you can manually set the :attr:`Response.encoding <requests.Response.encoding>` property, or use the raw :attr:`Response.content <requests.Response.content>`. .. _http-verbs: HTTP Verbs ---------- Requests provides access to almost the full range of HTTP verbs: GET, OPTIONS, HEAD, POST, PUT, PATCH and DELETE. The following provides detailed examples of using these various verbs in Requests, using the GitHub API. We will begin with the verb most commonly used: GET. HTTP GET is an idempotent method that returns a resource from a given URL. As a result, it is the verb you ought to use when attempting to retrieve data from a web location. An example usage would be attempting to get information about a specific commit from GitHub. Suppose we wanted commit ``a050faf`` on Requests. We would get it like so:: >>> import requests >>> r = requests.get('https://api.github.com/repos/psf/requests/git/commits/a050faf084662f3a352dd1a941f2c7c9f886d4ad') We should confirm that GitHub responded correctly. If it has, we want to work out what type of content it is. Do this like so:: >>> if r.status_code == requests.codes.ok: ... print(r.headers['content-type']) ... application/json; charset=utf-8 So, GitHub returns JSON. That's great, we can use the :meth:`r.json <requests.Response.json>` method to parse it into Python objects. :: >>> commit_data = r.json() >>> print(commit_data.keys()) ['committer', 'author', 'url', 'tree', 'sha', 'parents', 'message'] >>> print(commit_data['committer']) {'date': '2012-05-10T11:10:50-07:00', 'email': 'me@kennethreitz.com', 'name': 'Kenneth Reitz'} >>> print(commit_data['message']) makin' history So far, so simple. Well, let's investigate the GitHub API a little bit. Now, we could look at the documentation, but we might have a little more fun if we use Requests instead. We can take advantage of the Requests OPTIONS verb to see what kinds of HTTP methods are supported on the url we just used. :: >>> verbs = requests.options(r.url) >>> verbs.status_code 500 Uh, what? That's unhelpful! Turns out GitHub, like many API providers, don't actually implement the OPTIONS method. This is an annoying oversight, but it's OK, we can just use the boring documentation. If GitHub had correctly implemented OPTIONS, however, they should return the allowed methods in the headers, e.g. :: >>> verbs = requests.options('http://a-good-website.com/api/cats') >>> print(verbs.headers['allow']) GET,HEAD,POST,OPTIONS Turning to the documentation, we see that the only other method allowed for commits is POST, which creates a new commit. As we're using the Requests repo, we should probably avoid making ham-handed POSTS to it. Instead, let's play with the Issues feature of GitHub. This documentation was added in response to `Issue #482 <https://github.com/psf/requests/issues/482>`_. Given that this issue already exists, we will use it as an example. Let's start by getting it. :: >>> r = requests.get('https://api.github.com/repos/psf/requests/issues/482') >>> r.status_code 200 >>> issue = json.loads(r.text) >>> print(issue['title']) Feature any http verb in docs >>> print(issue['comments']) 3 Cool, we have three comments. Let's take a look at the last of them. :: >>> r = requests.get(r.url + '/comments') >>> r.status_code 200 >>> comments = r.json() >>> print(comments[0].keys()) ['body', 'url', 'created_at', 'updated_at', 'user', 'id'] >>> print(comments[2]['body']) Probably in the "advanced" section Well, that seems like a silly place. Let's post a comment telling the poster that he's silly. Who is the poster, anyway? :: >>> print(comments[2]['user']['login']) kennethreitz OK, so let's tell this Kenneth guy that we think this example should go in the quickstart guide instead. According to the GitHub API doc, the way to do this is to POST to the thread. Let's do it. :: >>> body = json.dumps({u"body": u"Sounds great! I'll get right on it!"}) >>> url = u"https://api.github.com/repos/psf/requests/issues/482/comments" >>> r = requests.post(url=url, data=body) >>> r.status_code 404 Huh, that's weird. We probably need to authenticate. That'll be a pain, right? Wrong. Requests makes it easy to use many forms of authentication, including the very common Basic Auth. :: >>> from requests.auth import HTTPBasicAuth >>> auth = HTTPBasicAuth('fake@example.com', 'not_a_real_password') >>> r = requests.post(url=url, data=body, auth=auth) >>> r.status_code 201 >>> content = r.json() >>> print(content['body']) Sounds great! I'll get right on it. Brilliant. Oh, wait, no! I meant to add that it would take me a while, because I had to go feed my cat. If only I could edit this comment! Happily, GitHub allows us to use another HTTP verb, PATCH, to edit this comment. Let's do that. :: >>> print(content[u"id"]) 5804413 >>> body = json.dumps({u"body": u"Sounds great! I'll get right on it once I feed my cat."}) >>> url = u"https://api.github.com/repos/psf/requests/issues/comments/5804413" >>> r = requests.patch(url=url, data=body, auth=auth) >>> r.status_code 200 Excellent. Now, just to torture this Kenneth guy, I've decided to let him sweat and not tell him that I'm working on this. That means I want to delete this comment. GitHub lets us delete comments using the incredibly aptly named DELETE method. Let's get rid of it. :: >>> r = requests.delete(url=url, auth=auth) >>> r.status_code 204 >>> r.headers['status'] '204 No Content' Excellent. All gone. The last thing I want to know is how much of my ratelimit I've used. Let's find out. GitHub sends that information in the headers, so rather than download the whole page I'll send a HEAD request to get the headers. :: >>> r = requests.head(url=url, auth=auth) >>> print(r.headers) ... 'x-ratelimit-remaining': '4995' 'x-ratelimit-limit': '5000' ... Excellent. Time to write a Python program that abuses the GitHub API in all kinds of exciting ways, 4995 more times. .. _custom-verbs: Custom Verbs ------------ From time to time you may be working with a server that, for whatever reason, allows use or even requires use of HTTP verbs not covered above. One example of this would be the MKCOL method some WEBDAV servers use. Do not fret, these can still be used with Requests. These make use of the built-in ``.request`` method. For example:: >>> r = requests.request('MKCOL', url, data=data) >>> r.status_code 200 # Assuming your call was correct Utilising this, you can make use of any method verb that your server allows. .. _link-headers: Link Headers ------------ Many HTTP APIs feature Link headers. They make APIs more self describing and discoverable. GitHub uses these for `pagination <https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api>`_ in their API, for example:: >>> url = 'https://api.github.com/users/kennethreitz/repos?page=1&per_page=10' >>> r = requests.head(url=url) >>> r.headers['link'] '<https://api.github.com/users/kennethreitz/repos?page=2&per_page=10>; rel="next", <https://api.github.com/users/kennethreitz/repos?page=6&per_page=10>; rel="last"' Requests will automatically parse these link headers and make them easily consumable:: >>> r.links["next"] {'url': 'https://api.github.com/users/kennethreitz/repos?page=2&per_page=10', 'rel': 'next'} >>> r.links["last"] {'url': 'https://api.github.com/users/kennethreitz/repos?page=7&per_page=10', 'rel': 'last'} .. _transport-adapters: Transport Adapters ------------------ As of v1.0.0, Requests has moved to a modular internal design. Part of the reason this was done was to implement Transport Adapters, originally `described here`_. Transport Adapters provide a mechanism to define interaction methods for an HTTP service. In particular, they allow you to apply per-service configuration. Requests ships with a single Transport Adapter, the :class:`HTTPAdapter <requests.adapters.HTTPAdapter>`. This adapter provides the default Requests interaction with HTTP and HTTPS using the powerful `urllib3`_ library. Whenever a Requests :class:`Session <requests.Session>` is initialized, one of these is attached to the :class:`Session <requests.Session>` object for HTTP, and one for HTTPS. Requests enables users to create and use their own Transport Adapters that provide specific functionality. Once created, a Transport Adapter can be mounted to a Session object, along with an indication of which web services it should apply to. :: >>> s = requests.Session() >>> s.mount('https://github.com/', MyAdapter()) The mount call registers a specific instance of a Transport Adapter to a prefix. Once mounted, any HTTP request made using that session whose URL starts with the given prefix will use the given Transport Adapter. .. note:: The adapter will be chosen based on a longest prefix match. Be mindful prefixes such as ``http://localhost`` will also match ``http://localhost.other.com`` or ``http://localhost@other.com``. It's recommended to terminate full hostnames with a ``/``. Many of the details of implementing a Transport Adapter are beyond the scope of this documentation, but take a look at the next example for a simple SSL use- case. For more than that, you might look at subclassing the :class:`BaseAdapter <requests.adapters.BaseAdapter>`. Example: Specific SSL Version ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The Requests team has made a specific choice to use whatever SSL version is default in the underlying library (`urllib3`_). Normally this is fine, but from time to time, you might find yourself needing to connect to a service-endpoint that uses a version that isn't compatible with the default. You can use Transport Adapters for this by taking most of the existing implementation of HTTPAdapter, and adding a parameter ssl_version that gets passed-through to `urllib3`. We'll make a Transport Adapter that instructs the library to use SSLv3:: import ssl from urllib3.poolmanager import PoolManager from requests.adapters import HTTPAdapter class Ssl3HttpAdapter(HTTPAdapter): """"Transport adapter" that allows us to use SSLv3.""" def init_poolmanager(self, connections, maxsize, block=False): self.poolmanager = PoolManager( num_pools=connections, maxsize=maxsize, block=block, ssl_version=ssl.PROTOCOL_SSLv3) Example: Automatic Retries ^^^^^^^^^^^^^^^^^^^^^^^^^^ By default, Requests does not retry failed connections. However, it is possible to implement automatic retries with a powerful array of features, including backoff, within a Requests :class:`Session <requests.Session>` using the `urllib3.util.Retry`_ class:: from urllib3.util import Retry from requests import Session from requests.adapters import HTTPAdapter s = Session() retries = Retry( total=3, backoff_factor=0.1, status_forcelist=[502, 503, 504], allowed_methods={'POST'}, ) s.mount('https://', HTTPAdapter(max_retries=retries)) .. _`described here`: https://kenreitz.org/essays/2012/06/14/the-future-of-python-http .. _`urllib3`: https://github.com/urllib3/urllib3 .. _`urllib3.util.Retry`: https://urllib3.readthedocs.io/en/stable/reference/urllib3.util.html#urllib3.util.Retry .. _blocking-or-nonblocking: Blocking Or Non-Blocking? ------------------------- With the default Transport Adapter in place, Requests does not provide any kind of non-blocking IO. The :attr:`Response.content <requests.Response.content>` property will block until the entire response has been downloaded. If you require more granularity, the streaming features of the library (see :ref:`streaming-requests`) allow you to retrieve smaller quantities of the response at a time. However, these calls will still block. If you are concerned about the use of blocking IO, there are lots of projects out there that combine Requests with one of Python's asynchronicity frameworks. Some excellent examples are `requests-threads`_, `grequests`_, `requests-futures`_, and `httpx`_. .. _`requests-threads`: https://github.com/requests/requests-threads .. _`grequests`: https://github.com/spyoungtech/grequests .. _`requests-futures`: https://github.com/ross/requests-futures .. _`httpx`: https://github.com/encode/httpx Header Ordering --------------- In unusual circumstances you may want to provide headers in an ordered manner. If you pass an ``OrderedDict`` to the ``headers`` keyword argument, that will provide the headers with an ordering. However, the ordering of the default headers used by Requests will be preferred, which means that if you override default headers in the ``headers`` keyword argument, they may appear out of order compared to other headers in that keyword argument. If this is problematic, users should consider setting the default headers on a :class:`Session <requests.Session>` object, by setting :attr:`Session.headers <requests.Session.headers>` to a custom ``OrderedDict``. That ordering will always be preferred. .. _timeouts: Timeouts -------- Most requests to external servers should have a timeout attached, in case the server is not responding in a timely manner. By default, requests do not time out unless a timeout value is set explicitly. Without a timeout, your code may hang for minutes or more. The connect timeout is the number of seconds Requests will wait for your client to establish a connection to a remote machine (corresponding to the `connect()`_) call on the socket. It's a good practice to set connect timeouts to slightly larger than a multiple of 3, which is the default `TCP packet retransmission window <https://datatracker.ietf.org/doc/html/rfc2988>`_. Once your client has connected to the server and sent the HTTP request, the read timeout is the number of seconds the client will wait for the server to send a response. (Specifically, it's the number of seconds that the client will wait between bytes sent from the server. In 99.9% of cases, this is the time before the server sends the first byte). If you specify a single value for the timeout, like this:: r = requests.get('https://github.com', timeout=5) The timeout value will be applied to both the ``connect`` and the ``read`` timeouts. Specify a tuple if you would like to set the values separately:: r = requests.get('https://github.com', timeout=(3.05, 27)) If the remote server is very slow, you can tell Requests to wait forever for a response, by passing None as a timeout value and then retrieving a cup of coffee. :: r = requests.get('https://github.com', timeout=None) .. note:: The connect timeout applies to each connection attempt to an IP address. If multiple addresses exist for a domain name, the underlying ``urllib3`` will try each address sequentially until one successfully connects. This may lead to an effective total connection timeout multiple times longer than the specified time, e.g. an unresponsive server having both IPv4 and IPv6 addresses will have its perceived timeout doubled, so take that into account when setting the connection timeout. .. note:: Neither the connect nor read timeouts are `wall clock`_. This means that if you start a request, and look at the time, and then look at the time when the request finishes or times out, the real-world time may be greater than what you specified. .. _`wall clock`: https://wiki.php.net/rfc/max_execution_wall_time .. _`connect()`: https://linux.die.net/man/2/connect	41.08 KB	2024-05-29 15:36
📄 authentication.rst .. _authentication: Authentication ============== This document discusses using various kinds of authentication with Requests. Many web services require authentication, and there are many different types. Below, we outline various forms of authentication available in Requests, from the simple to the complex. Basic Authentication -------------------- Many web services that require authentication accept HTTP Basic Auth. This is the simplest kind, and Requests supports it straight out of the box. Making requests with HTTP Basic Auth is very simple:: >>> from requests.auth import HTTPBasicAuth >>> basic = HTTPBasicAuth('user', 'pass') >>> requests.get('https://httpbin.org/basic-auth/user/pass', auth=basic) <Response [200]> In fact, HTTP Basic Auth is so common that Requests provides a handy shorthand for using it:: >>> requests.get('https://httpbin.org/basic-auth/user/pass', auth=('user', 'pass')) <Response [200]> Providing the credentials in a tuple like this is exactly the same as the ``HTTPBasicAuth`` example above. netrc Authentication ~~~~~~~~~~~~~~~~~~~~ If no authentication method is given with the ``auth`` argument, Requests will attempt to get the authentication credentials for the URL's hostname from the user's netrc file. The netrc file overrides raw HTTP authentication headers set with `headers=`. If credentials for the hostname are found, the request is sent with HTTP Basic Auth. Digest Authentication --------------------- Another very popular form of HTTP Authentication is Digest Authentication, and Requests supports this out of the box as well:: >>> from requests.auth import HTTPDigestAuth >>> url = 'https://httpbin.org/digest-auth/auth/user/pass' >>> requests.get(url, auth=HTTPDigestAuth('user', 'pass')) <Response [200]> OAuth 1 Authentication ---------------------- A common form of authentication for several web APIs is OAuth. The ``requests-oauthlib`` library allows Requests users to easily make OAuth 1 authenticated requests:: >>> import requests >>> from requests_oauthlib import OAuth1 >>> url = 'https://api.twitter.com/1.1/account/verify_credentials.json' >>> auth = OAuth1('YOUR_APP_KEY', 'YOUR_APP_SECRET', ... 'USER_OAUTH_TOKEN', 'USER_OAUTH_TOKEN_SECRET') >>> requests.get(url, auth=auth) <Response [200]> For more information on how to OAuth flow works, please see the official `OAuth`_ website. For examples and documentation on requests-oauthlib, please see the `requests_oauthlib`_ repository on GitHub OAuth 2 and OpenID Connect Authentication ----------------------------------------- The ``requests-oauthlib`` library also handles OAuth 2, the authentication mechanism underpinning OpenID Connect. See the `requests-oauthlib OAuth2 documentation`_ for details of the various OAuth 2 credential management flows: * `Web Application Flow`_ * `Mobile Application Flow`_ * `Legacy Application Flow`_ * `Backend Application Flow`_ Other Authentication -------------------- Requests is designed to allow other forms of authentication to be easily and quickly plugged in. Members of the open-source community frequently write authentication handlers for more complicated or less commonly-used forms of authentication. Some of the best have been brought together under the `Requests organization`_, including: - Kerberos_ - NTLM_ If you want to use any of these forms of authentication, go straight to their GitHub page and follow the instructions. New Forms of Authentication --------------------------- If you can't find a good implementation of the form of authentication you want, you can implement it yourself. Requests makes it easy to add your own forms of authentication. To do so, subclass :class:`AuthBase <requests.auth.AuthBase>` and implement the ``__call__()`` method:: >>> import requests >>> class MyAuth(requests.auth.AuthBase): ... def __call__(self, r): ... # Implement my authentication ... return r ... >>> url = 'https://httpbin.org/get' >>> requests.get(url, auth=MyAuth()) <Response [200]> When an authentication handler is attached to a request, it is called during request setup. The ``__call__`` method must therefore do whatever is required to make the authentication work. Some forms of authentication will additionally add hooks to provide further functionality. Further examples can be found under the `Requests organization`_ and in the ``auth.py`` file. .. _OAuth: https://oauth.net/ .. _requests_oauthlib: https://github.com/requests/requests-oauthlib .. _requests-oauthlib OAuth2 documentation: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html .. _Web Application Flow: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#web-application-flow .. _Mobile Application Flow: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#mobile-application-flow .. _Legacy Application Flow: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#legacy-application-flow .. _Backend Application Flow: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#backend-application-flow .. _Kerberos: https://github.com/requests/requests-kerberos .. _NTLM: https://github.com/requests/requests-ntlm .. _Requests organization: https://github.com/requests	5.27 KB	2024-05-29 15:36
📄 install.rst .. _install: Installation of Requests ======================== This part of the documentation covers the installation of Requests. The first step to using any software package is getting it properly installed. $ python -m pip install requests -------------------------------- To install Requests, simply run this simple command in your terminal of choice:: $ python -m pip install requests Get the Source Code ------------------- Requests is actively developed on GitHub, where the code is `always available <https://github.com/psf/requests>`_. You can either clone the public repository:: $ git clone https://github.com/psf/requests.git Or, download the `tarball <https://github.com/psf/requests/tarball/main>`_:: $ curl -OL https://github.com/psf/requests/tarball/main # optionally, zipball is also available (for Windows users). Once you have a copy of the source, you can embed it in your own Python package, or install it into your site-packages easily:: $ cd requests $ python -m pip install .	1.01 KB	2024-05-29 15:36
📄 quickstart.rst .. _quickstart: Quickstart ========== .. module:: requests.models Eager to get started? This page gives a good introduction in how to get started with Requests. First, make sure that: * Requests is :ref:`installed <install>` * Requests is :ref:`up-to-date <updates>` Let's get started with some simple examples. Make a Request -------------- Making a request with Requests is very simple. Begin by importing the Requests module:: >>> import requests Now, let's try to get a webpage. For this example, let's get GitHub's public timeline:: >>> r = requests.get('https://api.github.com/events') Now, we have a :class:`Response <requests.Response>` object called ``r``. We can get all the information we need from this object. Requests' simple API means that all forms of HTTP request are as obvious. For example, this is how you make an HTTP POST request:: >>> r = requests.post('https://httpbin.org/post', data={'key': 'value'}) Nice, right? What about the other HTTP request types: PUT, DELETE, HEAD and OPTIONS? These are all just as simple:: >>> r = requests.put('https://httpbin.org/put', data={'key': 'value'}) >>> r = requests.delete('https://httpbin.org/delete') >>> r = requests.head('https://httpbin.org/get') >>> r = requests.options('https://httpbin.org/get') That's all well and good, but it's also only the start of what Requests can do. Passing Parameters In URLs -------------------------- You often want to send some sort of data in the URL's query string. If you were constructing the URL by hand, this data would be given as key/value pairs in the URL after a question mark, e.g. ``httpbin.org/get?key=val``. Requests allows you to provide these arguments as a dictionary of strings, using the ``params`` keyword argument. As an example, if you wanted to pass ``key1=value1`` and ``key2=value2`` to ``httpbin.org/get``, you would use the following code:: >>> payload = {'key1': 'value1', 'key2': 'value2'} >>> r = requests.get('https://httpbin.org/get', params=payload) You can see that the URL has been correctly encoded by printing the URL:: >>> print(r.url) https://httpbin.org/get?key2=value2&key1=value1 Note that any dictionary key whose value is ``None`` will not be added to the URL's query string. You can also pass a list of items as a value:: >>> payload = {'key1': 'value1', 'key2': ['value2', 'value3']} >>> r = requests.get('https://httpbin.org/get', params=payload) >>> print(r.url) https://httpbin.org/get?key1=value1&key2=value2&key2=value3 Response Content ---------------- We can read the content of the server's response. Consider the GitHub timeline again:: >>> import requests >>> r = requests.get('https://api.github.com/events') >>> r.text '[{"repository":{"open_issues":0,"url":"https://github.com/... Requests will automatically decode content from the server. Most unicode charsets are seamlessly decoded. When you make a request, Requests makes educated guesses about the encoding of the response based on the HTTP headers. The text encoding guessed by Requests is used when you access ``r.text``. You can find out what encoding Requests is using, and change it, using the ``r.encoding`` property:: >>> r.encoding 'utf-8' >>> r.encoding = 'ISO-8859-1' If you change the encoding, Requests will use the new value of ``r.encoding`` whenever you call ``r.text``. You might want to do this in any situation where you can apply special logic to work out what the encoding of the content will be. For example, HTML and XML have the ability to specify their encoding in their body. In situations like this, you should use ``r.content`` to find the encoding, and then set ``r.encoding``. This will let you use ``r.text`` with the correct encoding. Requests will also use custom encodings in the event that you need them. If you have created your own encoding and registered it with the ``codecs`` module, you can simply use the codec name as the value of ``r.encoding`` and Requests will handle the decoding for you. Binary Response Content ----------------------- You can also access the response body as bytes, for non-text requests:: >>> r.content b'[{"repository":{"open_issues":0,"url":"https://github.com/... The ``gzip`` and ``deflate`` transfer-encodings are automatically decoded for you. The ``br`` transfer-encoding is automatically decoded for you if a Brotli library like `brotli <https://pypi.org/project/brotli>`_ or `brotlicffi <https://pypi.org/project/brotlicffi>`_ is installed. For example, to create an image from binary data returned by a request, you can use the following code:: >>> from PIL import Image >>> from io import BytesIO >>> i = Image.open(BytesIO(r.content)) JSON Response Content --------------------- There's also a builtin JSON decoder, in case you're dealing with JSON data:: >>> import requests >>> r = requests.get('https://api.github.com/events') >>> r.json() [{'repository': {'open_issues': 0, 'url': 'https://github.com/... In case the JSON decoding fails, ``r.json()`` raises an exception. For example, if the response gets a 204 (No Content), or if the response contains invalid JSON, attempting ``r.json()`` raises ``requests.exceptions.JSONDecodeError``. This wrapper exception provides interoperability for multiple exceptions that may be thrown by different python versions and json serialization libraries. It should be noted that the success of the call to ``r.json()`` does not indicate the success of the response. Some servers may return a JSON object in a failed response (e.g. error details with HTTP 500). Such JSON will be decoded and returned. To check that a request is successful, use ``r.raise_for_status()`` or check ``r.status_code`` is what you expect. Raw Response Content -------------------- In the rare case that you'd like to get the raw socket response from the server, you can access ``r.raw``. If you want to do this, make sure you set ``stream=True`` in your initial request. Once you do, you can do this:: >>> r = requests.get('https://api.github.com/events', stream=True) >>> r.raw <urllib3.response.HTTPResponse object at 0x101194810> >>> r.raw.read(10) b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03' In general, however, you should use a pattern like this to save what is being streamed to a file:: with open(filename, 'wb') as fd: for chunk in r.iter_content(chunk_size=128): fd.write(chunk) Using ``Response.iter_content`` will handle a lot of what you would otherwise have to handle when using ``Response.raw`` directly. When streaming a download, the above is the preferred and recommended way to retrieve the content. Note that ``chunk_size`` can be freely adjusted to a number that may better fit your use cases. .. note:: An important note about using ``Response.iter_content`` versus ``Response.raw``. ``Response.iter_content`` will automatically decode the ``gzip`` and ``deflate`` transfer-encodings. ``Response.raw`` is a raw stream of bytes -- it does not transform the response content. If you really need access to the bytes as they were returned, use ``Response.raw``. .. _custom-headers: Custom Headers -------------- If you'd like to add HTTP headers to a request, simply pass in a ``dict`` to the ``headers`` parameter. For example, we didn't specify our user-agent in the previous example:: >>> url = 'https://api.github.com/some/endpoint' >>> headers = {'user-agent': 'my-app/0.0.1'} >>> r = requests.get(url, headers=headers) Note: Custom headers are given less precedence than more specific sources of information. For instance: * Authorization headers set with `headers=` will be overridden if credentials are specified in ``.netrc``, which in turn will be overridden by the ``auth=`` parameter. Requests will search for the netrc file at `~/.netrc`, `~/_netrc`, or at the path specified by the `NETRC` environment variable. * Authorization headers will be removed if you get redirected off-host. * Proxy-Authorization headers will be overridden by proxy credentials provided in the URL. * Content-Length headers will be overridden when we can determine the length of the content. Furthermore, Requests does not change its behavior at all based on which custom headers are specified. The headers are simply passed on into the final request. Note: All header values must be a ``string``, bytestring, or unicode. While permitted, it's advised to avoid passing unicode header values. More complicated POST requests ------------------------------ Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the ``data`` argument. Your dictionary of data will automatically be form-encoded when the request is made:: >>> payload = {'key1': 'value1', 'key2': 'value2'} >>> r = requests.post('https://httpbin.org/post', data=payload) >>> print(r.text) { ... "form": { "key2": "value2", "key1": "value1" }, ... } The ``data`` argument can also have multiple values for each key. This can be done by making ``data`` either a list of tuples or a dictionary with lists as values. This is particularly useful when the form has multiple elements that use the same key:: >>> payload_tuples = [('key1', 'value1'), ('key1', 'value2')] >>> r1 = requests.post('https://httpbin.org/post', data=payload_tuples) >>> payload_dict = {'key1': ['value1', 'value2']} >>> r2 = requests.post('https://httpbin.org/post', data=payload_dict) >>> print(r1.text) { ... "form": { "key1": [ "value1", "value2" ] }, ... } >>> r1.text == r2.text True There are times that you may want to send data that is not form-encoded. If you pass in a ``string`` instead of a ``dict``, that data will be posted directly. For example, the GitHub API v3 accepts JSON-Encoded POST/PATCH data:: >>> import json >>> url = 'https://api.github.com/some/endpoint' >>> payload = {'some': 'data'} >>> r = requests.post(url, data=json.dumps(payload)) Please note that the above code will NOT add the ``Content-Type`` header (so in particular it will NOT set it to ``application/json``). If you need that header set and you don't want to encode the ``dict`` yourself, you can also pass it directly using the ``json`` parameter (added in version 2.4.2) and it will be encoded automatically: >>> url = 'https://api.github.com/some/endpoint' >>> payload = {'some': 'data'} >>> r = requests.post(url, json=payload) Note, the ``json`` parameter is ignored if either ``data`` or ``files`` is passed. POST a Multipart-Encoded File ----------------------------- Requests makes it simple to upload Multipart-encoded files:: >>> url = 'https://httpbin.org/post' >>> files = {'file': open('report.xls', 'rb')} >>> r = requests.post(url, files=files) >>> r.text { ... "files": { "file": "<censored...binary...data>" }, ... } You can set the filename, content_type and headers explicitly:: >>> url = 'https://httpbin.org/post' >>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})} >>> r = requests.post(url, files=files) >>> r.text { ... "files": { "file": "<censored...binary...data>" }, ... } If you want, you can send strings to be received as files:: >>> url = 'https://httpbin.org/post' >>> files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')} >>> r = requests.post(url, files=files) >>> r.text { ... "files": { "file": "some,data,to,send\\nanother,row,to,send\\n" }, ... } In the event you are posting a very large file as a ``multipart/form-data`` request, you may want to stream the request. By default, ``requests`` does not support this, but there is a separate package which does - ``requests-toolbelt``. You should read `the toolbelt's documentation <https://toolbelt.readthedocs.io>`_ for more details about how to use it. For sending multiple files in one request refer to the :ref:`advanced <advanced>` section. .. warning:: It is strongly recommended that you open files in :ref:`binary mode <tut-files>`. This is because Requests may attempt to provide the ``Content-Length`` header for you, and if it does this value will be set to the number of bytes in the file. Errors may occur if you open the file in text mode. Response Status Codes --------------------- We can check the response status code:: >>> r = requests.get('https://httpbin.org/get') >>> r.status_code 200 Requests also comes with a built-in status code lookup object for easy reference:: >>> r.status_code == requests.codes.ok True If we made a bad request (a 4XX client error or 5XX server error response), we can raise it with :meth:`Response.raise_for_status() <requests.Response.raise_for_status>`:: >>> bad_r = requests.get('https://httpbin.org/status/404') >>> bad_r.status_code 404 >>> bad_r.raise_for_status() Traceback (most recent call last): File "requests/models.py", line 832, in raise_for_status raise http_error requests.exceptions.HTTPError: 404 Client Error But, since our ``status_code`` for ``r`` was ``200``, when we call ``raise_for_status()`` we get:: >>> r.raise_for_status() None All is well. Response Headers ---------------- We can view the server's response headers using a Python dictionary:: >>> r.headers { 'content-encoding': 'gzip', 'transfer-encoding': 'chunked', 'connection': 'close', 'server': 'nginx/1.0.4', 'x-runtime': '148ms', 'etag': '"e1ca502697e5c9317743dc078f67693f"', 'content-type': 'application/json' } The dictionary is special, though: it's made just for HTTP headers. According to `RFC 7230 <https://tools.ietf.org/html/rfc7230#section-3.2>`_, HTTP Header names are case-insensitive. So, we can access the headers using any capitalization we want:: >>> r.headers['Content-Type'] 'application/json' >>> r.headers.get('content-type') 'application/json' It is also special in that the server could have sent the same header multiple times with different values, but requests combines them so they can be represented in the dictionary within a single mapping, as per `RFC 7230 <https://tools.ietf.org/html/rfc7230#section-3.2>`_: A recipient MAY combine multiple header fields with the same field name into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field value to the combined field value in order, separated by a comma. Cookies ------- If a response contains some Cookies, you can quickly access them:: >>> url = 'http://example.com/some/cookie/setting/url' >>> r = requests.get(url) >>> r.cookies['example_cookie_name'] 'example_cookie_value' To send your own cookies to the server, you can use the ``cookies`` parameter:: >>> url = 'https://httpbin.org/cookies' >>> cookies = dict(cookies_are='working') >>> r = requests.get(url, cookies=cookies) >>> r.text '{"cookies": {"cookies_are": "working"}}' Cookies are returned in a :class:`~requests.cookies.RequestsCookieJar`, which acts like a ``dict`` but also offers a more complete interface, suitable for use over multiple domains or paths. Cookie jars can also be passed in to requests:: >>> jar = requests.cookies.RequestsCookieJar() >>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies') >>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere') >>> url = 'https://httpbin.org/cookies' >>> r = requests.get(url, cookies=jar) >>> r.text '{"cookies": {"tasty_cookie": "yum"}}' Redirection and History ----------------------- By default Requests will perform location redirection for all verbs except HEAD. We can use the ``history`` property of the Response object to track redirection. The :attr:`Response.history <requests.Response.history>` list contains the :class:`Response <requests.Response>` objects that were created in order to complete the request. The list is sorted from the oldest to the most recent response. For example, GitHub redirects all HTTP requests to HTTPS:: >>> r = requests.get('http://github.com/') >>> r.url 'https://github.com/' >>> r.status_code 200 >>> r.history [<Response [301]>] If you're using GET, OPTIONS, POST, PUT, PATCH or DELETE, you can disable redirection handling with the ``allow_redirects`` parameter:: >>> r = requests.get('http://github.com/', allow_redirects=False) >>> r.status_code 301 >>> r.history [] If you're using HEAD, you can enable redirection as well:: >>> r = requests.head('http://github.com/', allow_redirects=True) >>> r.url 'https://github.com/' >>> r.history [<Response [301]>] Timeouts -------- You can tell Requests to stop waiting for a response after a given number of seconds with the ``timeout`` parameter. Nearly all production code should use this parameter in nearly all requests. Failure to do so can cause your program to hang indefinitely:: >>> requests.get('https://github.com/', timeout=0.001) Traceback (most recent call last): File "<stdin>", line 1, in <module> requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001) .. admonition:: Note ``timeout`` is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for ``timeout`` seconds (more precisely, if no bytes have been received on the underlying socket for ``timeout`` seconds). If no timeout is specified explicitly, requests do not time out. Errors and Exceptions --------------------- In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a :exc:`~requests.exceptions.ConnectionError` exception. :meth:`Response.raise_for_status() <requests.Response.raise_for_status>` will raise an :exc:`~requests.exceptions.HTTPError` if the HTTP request returned an unsuccessful status code. If a request times out, a :exc:`~requests.exceptions.Timeout` exception is raised. If a request exceeds the configured number of maximum redirections, a :exc:`~requests.exceptions.TooManyRedirects` exception is raised. All exceptions that Requests explicitly raises inherit from :exc:`requests.exceptions.RequestException`. ----------------------- Ready for more? Check out the :ref:`advanced <advanced>` section.	18.57 KB	2024-05-29 15:36

Terminal