Ձj) d Z ddlZddlmZmZ ddlmZ dxZZddd d
d d d
dZ i Z
d Z d Z d Z
G d d Z e d Z e d Z e d Zd#dZd ZedfdZd#dZd Zedfd Zd Z G d d Z G d! d" Zy)$a
webencodings
~~~~~~~~~~~~
This is a Python implementation of the `WHATWG Encoding standard
`. See README for details.
:copyright: Copyright 2012 by Simon Sapin
:license: BSD, see LICENSE for details.
N )replacement_codec_infouser_codec_info)LABELSz0.6.1z iso8859-8z mac-cyrillicz mac-romancp874cp932 big5hkscscp949)z iso-8859-8-izx-mac-cyrillic macintoshz windows-874 shift_jisbig5zeuc-krc Z | j j j S )a2 Transform (only) ASCII letters to lower case: A-Z is mapped to a-z.
:param string: An Unicode string.
:returns: A new Unicode string.
This is used for `ASCII case-insensitive
`_
matching of encoding labels.
The same matching is also used, among other things,
for `CSS keywords `_.
This is different from the :meth:`str.lower` method of Unicode strings
which also affect non-ASCII characters,
sometimes mapping them into the ASCII range:
>>> keyword = 'Bac\N{KELVIN SIGN}ground'
>>> assert keyword.lower() == 'background'
>>> assert ascii_lower(keyword) != keyword.lower()
>>> assert ascii_lower(keyword) == 'bac\N{KELVIN SIGN}ground'
)encodelowerdecode)strings `_ algorithm.
Supported labels are listed there.
:param label: A string.
:returns:
An :class:`Encoding` object, or :obj:`None` for an unknown label.
z
Nzx-user-defined replacement) r stripr getCACHEr r PYTHON_NAMEScodecslookupEncoding)labelnameencoding
codec_info python_names r r r @ s
K0 1E ::e D |yyH # #(J
]
"/J&**46K {3JD*- d Or c X t | d r| S t | }|t d| |S )zAccept either an encoding object or label.
:param encoding: An :class:`Encoding` object or a label string.
:returns: An :class:`Encoding` object.
:raises: :exc:`LookupError` for an unknown label.
r" zUnknown encoding label: )hasattrr LookupError)encoding_or_labelr! s r
_get_encodingr( ` s@ ,/ '(H 45F4IJKK Or c e Zd ZdZd Zd Zy)r a2 A character encoding that can be used for decoding or encoding.
.. attribute:: name
Canonical name of the encoding
.. attribute:: codec_info
The actual implementation of the encoding,
a stdlib :class:`~codecs.CodecInfo` object.
See :func:`codecs.register`.
c || _ || _ y N)r r" )selfr r" s r __init__zEncoding.__init__ s $r c " d| j dS )Nz
)r )r, s r __repr__zEncoding.__repr__ s DII;a((r N)__name__
__module__ __qualname____doc__r- r0 r r r r q s %)r r zutf-8zutf-16lezutf-16bec t | }t | \ }} |xs |}|j j | | d |fS )a Decode a single string.
:param input: A byte string
:param fallback_encoding:
An :class:`Encoding` object or a label string.
The encoding to use if :obj:`input` does not have a BOM.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`LookupError` for an unknown encoding label.
:return:
A ``(output, encoding)`` tuple of an Unicode string
and an :obj:`Encoding`.
r )r( _detect_bomr" r )inputfallback_encodingerrors bom_encodingr! s r r r sM &&78%e,L%0 0H % %eV 4Q 7 AAr c | j d r t | dd fS | j d r t | dd fS | j d r t | dd fS d| fS )zBReturn (bom_encoding, input), with any BOM removed from the input.s Ns s )
startswith_UTF16LE_UTF16BEUTF8)r8 s r r7 r7 sh $qr"" $qr"" (U12Y ;r strictc R t | j j | | d S )a* Encode a single string.
:param input: An Unicode string.
:param encoding: An :class:`Encoding` object or a label string.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`LookupError` for an unknown encoding label.
:return: A byte string.
r )r( r" r )r8 r! r: s r r r s' " - - 4 4UF CA FFr c P t || }t | | }t | }||fS )a "Pull"-based decoder.
:param input:
An iterable of byte strings.
The input is first consumed just enough to determine the encoding
based on the precense of a BOM,
then consumed on demand when the return value is.
:param fallback_encoding:
An :class:`Encoding` object or a label string.
The encoding to use if :obj:`input` does not have a BOM.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`LookupError` for an unknown encoding label.
:returns:
An ``(output, encoding)`` tuple.
``output`` is an iterable of Unicode strings,
``encoding`` is the :obj:`Encoding` that is being used.
)IncrementalDecoder_iter_decode_generatornext)r8 r9 r: decoder generatorr! s r iter_decoderK s1 * !!2F;G&ug6IIH h r c # H K |j }t | } | D ]. } || }|s|j J |j | n. |dd }|j J |j |r| y| D ] } || }|s| |dd }|r| yyw)z|Return a decode generator.
It first yields the :obj:`Encoding`, then yields output chunks as Unicode
strings.
Nr Tfinal)r iterr! )r8 rI r chunckoutputs r rG rG s ^^F KE ##/ //"" "L
4( +++ L L Ct
$F
s 'B"A B"
B"c F t || j }t | | S )aD "Pull"-based encoder.
:param input: An iterable of Unicode strings.
:param encoding: An :class:`Encoding` object or a label string.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`LookupError` for an unknown encoding label.
:returns: An iterable of byte strings.
)IncrementalEncoderr _iter_encode_generator)r8 r! r: r s r iter_encoderU s# &
1
8
8F !% 00r c # Z K | D ] } || }|s| |dd }|r| yyw)zReturn an encode generator. TrM Nr5 )r8 r rP rQ s r rT rT sA L Bd
#F
s ++c e Zd ZdZddZddZy)rF a9 "Push"-based decoder.
:param fallback_encoding:
An :class:`Encoding` object or a label string.
The encoding to use if :obj:`input` does not have a BOM.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`LookupError` for an unknown encoding label.
c \ t | | _ || _ d| _ d | _ d | _ y )Nr )r( _fallback_encoding_errors_buffer_decoderr! )r, r9 r: s r r- zIncrementalDecoder.__init__ s. "/0A"B
r c B | j }| ||| S | j |z }t | \ }}|$t | dk r
|s|| _ y| j }|j
j
| j j }|| _ || _ ||| S )zDecode one chunk of the input.
:param input: A byte string.
:param final:
Indicate that no more input is available.
Must be :obj:`True` if this is the last call.
:returns: An Unicode string.
r> rW )
r] r\ r7 lenrZ r" incrementaldecoderr[ r r! )r, r8 rN rI r! s r r zIncrementalDecoder.decode( s -- 5%( ( u$%e,% 5zA~e $ 22%%88 FMM
ue$$r Nreplace)F)r1 r2 r3 r4 r- r r5 r r rF rF s
%r rF c e Zd ZdZedfdZy)rS a "Push"-based encoder.
:param encoding: An :class:`Encoding` object or a label string.
:param errors: Type of error handling. See :func:`codecs.register`.
:raises: :exc:`LookupError` for an unknown encoding label.
.. method:: encode(input, final=False)
:param input: An Unicode string.
:param final:
Indicate that no more input is available.
Must be :obj:`True` if this is the last call.
:returns: A byte string.
rC c n t | }|j j | j | _ y r+ )r( r" incrementalencoderr )r, r! r: s r r- zIncrementalEncoder.__init__T s* * ))<rj s ; +
$ ,4 @") ). g * * B*
G 6 D !%X 1 /% /%dL Lr