Authentication and Cookies

Posted: 4 days ago Quote #4
Do you have an sampe whichs demonstrates an authentication against a website and uses the authenticated user (cookies) for all following requests?
Posted: 4 days ago Quote #5
I assume you mean "Forms Authentication" and sessions with cookies.
The HttpRequest workflow element supports both.

Cookies
You must set a CookieContainer in the CookieContainer property of the HttpRequestConfig to propagate cookies from one request to another.

Forms
Forms authentication are http posts of form data. Posts can be done with the PostRequest property of the  HttpRequestConfig. There is a HtmlForm class witch helps you to manage forms and you can easily fetch the forms on a HTML page via the Forms Property of the HttpRequest. and you can get the URL encoded form data with the method GetUrlEncodedFormData() out of it. You just transfer this to binary with an encoding (typically that encoding the site uses) and pack it in a post request.

Pitfalls with Cookies and Forms Authentication
If you use the proxy and datacenter features (HttpProxyDatacenterAccessPath) keep in mind that your CookieContainer is only valid for a certain proxy and datacenter pair. You should drive all requests through this proxy-datacenter pair. If you use the Retry workflow element to retry the operation on a different proxy-datacenter combination, you must gather the HttpProxyDatacenterAccessPath on the beginning of the StartWork handler of the Retry workflow element and use it for all requests inside.

Parallel requests (all HttpRequests created in a handler are parallel by default) need an own CookieContainer.
Strictly sequence your workflow ( start the next request in the success handler of the previous) when you use CookieContainer objects.

Last but not least I will provide a sample for this scenario with the next release of the crawler lib engine. The engine will be released in the next days and brings all the cool async/await stuff with it. The samples will be updated shortly after the release.    
Crawler-Lib Developer