Server : Apache System : Linux server1.cgrithy.com 3.10.0-1160.95.1.el7.x86_64 #1 SMP Mon Jul 24 13:59:37 UTC 2023 x86_64 User : nobody ( 99) PHP Version : 8.1.23 Disable Function : NONE Directory : /opt/alt/python311/lib64/python3.11/urllib/__pycache__/ |
� �,d�$ � � � d dl Z d dlZd dlZdgZ e j dd� � Z G d� d� � Z G d� d� � Z G d� d � � Z dS ) � N�RobotFileParser�RequestRatezrequests secondsc �Z � e Zd Z dd�Zd� Zd� Zd� Zd� Zd� Zd� Z d � Z d � Zd� Zd� Z d � ZdS )r � c � � g | _ g | _ d | _ d| _ d| _ | � |� � d| _ d S )NFr )�entries�sitemaps� default_entry�disallow_all� allow_all�set_url�last_checked��self�urls �9/opt/alt/python311/lib64/python3.11/urllib/robotparser.py�__init__zRobotFileParser.__init__ sG � ������ �!���!���������S��������� c � � | j S �N)r �r s r �mtimezRobotFileParser.mtime% s � � � � � r c �B � dd l }|� � � | _ d S )Nr )�timer )r r s r �modifiedzRobotFileParser.modified. s( � � � ���� �I�I�K�K����r c �~ � || _ t j � |� � dd� \ | _ | _ d S )N� � )r �urllib�parse�urlparse�host�pathr s r r zRobotFileParser.set_url6 s7 � �:����%�|�4�4�S�9�9�!�A�#�>��� �4�9�9�9r c � � t j � | j � � }|� � � }| � |� d� � � � � � � d S # t j j $ rK}|j dv rd| _ n)|j dk r|j dk rd| _ Y d }~d S Y d }~d S Y d }~d S Y d }~d S d }~ww xY w)Nzutf-8)i� i� Ti� i� ) r �request�urlopenr �readr �decode� splitlines�error� HTTPError�coder r )r �f�raw�errs r r'