Skip to main content

Allow access to secured websites

Give the website crawler a unique header (or equivalent) so it can be identified and any authentication can be bypassed on secured sites

Log in to comment and vote

No comments yet

Be the first to share your thoughts.