File: //opt/alt/python311/lib/python3.11/site-packages/chardet/__pycache__/charsetprober.cpython-311.pyc
�
��;f� � �> � d dl Z d dlZddlmZ G d� de� � ZdS )� N� )� ProbingStatec � � e Zd ZdZd d�Zd� Zed� � � Zd� Zed� � � Z d� Z
e d � � � Z e d
� � � Z
e d � � � ZdS )
�
CharSetProbergffffff�?Nc �^ � d | _ || _ t j t � � | _ d S �N)�_state� lang_filter�logging� getLogger�__name__�logger)�selfr
s �H/opt/alt/python311/lib/python3.11/site-packages/chardet/charsetprober.py�__init__zCharSetProber.__init__' s' � ��� �&��� �'��1�1�� � � � c �( � t j | _ d S r )r � DETECTINGr �r s r �resetzCharSetProber.reset, s � �"�,�� � � r c � � d S r � r s r � charset_namezCharSetProber.charset_name/ s � ��tr c � � d S r r )r �bufs r �feedzCharSetProber.feed3 s � � �r c � � | j S r )r r s r �statezCharSetProber.state6 s
� ��{�r c � � dS )Ng r r s r �get_confidencez CharSetProber.get_confidence: s � ��sr c �2 � t j dd| � � } | S )Ns ([ -])+� )�re�sub)r s r �filter_high_byte_onlyz#CharSetProber.filter_high_byte_only= s � ��f�&��c�2�2���
r c � � t � � }t j d| � � }|D ]Z}|� |dd� � � |dd� }|� � � s|dk rd}|� |� � �[|S )u9
We define three types of bytes:
alphabet: english alphabets [a-zA-Z]
international: international characters [-ÿ]
marker: everything else [^a-zA-Z-ÿ]
The input buffer can be thought to contain a series of words delimited
by markers. This function works to filter all words that contain at
least one international character. All contiguous sequences of markers
are replaced by a single space ascii character.
This filter applies to all scripts which do not use English characters.
s% [a-zA-Z]*[�-�]+[a-zA-Z]*[^a-zA-Z�-�]?N���� �r" )� bytearrayr# �findall�extend�isalpha)r �filtered�words�word� last_chars r �filter_international_wordsz(CharSetProber.filter_international_wordsB s� � � �;�;��
�
�O� � � �� �
'�
'�D� �O�O�D��"��I� &� &� &� �R�S�S� �I� �$�$�&�&�
!�9�w�+>�+>� � � �O�O�I� &� &� &� &��r c � � t � � }d}d}t t | � � � � D ]y}| ||dz � }|dk rd}n|dk rd}|dk rS|� � � s?||k r4|s2|� | ||� � � |� d� � |dz }�z|s |� | |d � � � |S )
a�
Returns a copy of ``buf`` that retains only the sequences of English
alphabet and high byte characters that are not between <> characters.
Also retains English alphabet and high byte characters immediately
before occurrences of >.
This filter can be applied to all scripts which contain both English
characters and extended ASCII characters, but is currently only used by
``Latin1Prober``.
Fr r � >� <Tr( r" N)r) �range�lenr, r+ )r r- �in_tag�prev�curr�buf_chars r �filter_with_english_lettersz)CharSetProber.filter_with_english_lettersg s� � � �;�;�������#�c�(�(�O�O� � �D��4��q��=�)�H��4� � � ����T�!�!� �� �'�!�!�(�*:�*:�*<�*<�!��$�;�;�v�;� �O�O�C��T� �N�3�3�3� �O�O�D�)�)�)��a�x��� � (�
�O�O�C����J� '� '� '��r r )r
�
__module__� __qualname__�SHORTCUT_THRESHOLDr r �propertyr r r r � staticmethodr% r1 r; r r r r r # s� � � � � � � ��2� 2� 2� 2�
-� -� -� �� � �X��
�
�
� �� � �X��� � � �� � �\�� �"� "� �\�"�H �)� )� �\�)� )� )r r )r r# �enumsr �objectr r r r �<module>rC si ��: ���� � � � � � � � � � �n� n� n� n� n�F� n� n� n� n� nr