KWTS 6.1-ISO. Integration with external ICAP service [Kaspersky Web Traffic Security]

[Antipova Anna]

Advice and Solutions (Forum Knowledgebase) Disclaimer. Read before using materials.

General information

Kaspersky Web Traffic Security does not have a regular function of integration with external services via the ICAP protocol, however, it can be added by manually changing the configuration files of the built-in proxy server from the technical support mode.

Important: ICAP integration works in synchronous mode - data transfer is suspended until the ICAP server processes the request. This may introduce additional delays in the processing of user traffic, thus reducing the performance of the proxy server. The external ICAP service must be able to process a sufficient number of concurrent requests and be designed for the target load according to the manufacturer's recommendations.

Integration options

Below are several configuration options depending on which data streams you want to pass through an external ICAP service. To reduce the load, additional filtering of requests using ACL rules is possible. The configuration fragment for the selected integration option must be added to the built-in proxy server configuration file template according to the instructions at the end of the article.

In the examples, the chain of ICAP services is built in such a way that first the request is sent to an external ICAP, and secondly it is checked against KWTS. If necessary, the order can be changed by changing the adaptation_service_chain directive accordingly.

The address of the ICAP service and the method of interaction with it are determined by the icap_service directive:

icap_service is_ext_req reqmod_precache bypass=0 icap://IPADDRESS:PORT/PATH

bypass - determines how the proxy server will behave when the service is unavailable:

• bypass=0 - the service is required and if it is unavailable, the user will see an error instead of the requested page
• bypass=1 - if the service is unavailable, it will be skipped

icap://IPADDRESS:PORT/PATH - ICAP service address:

• IPADDRESS - service IP address (domain name cannot be specified)
• PORT - TCP port number
• PATH - path to the service (check the value in the documentation for the service)

Option 1. Sending only HTTP requests to external ICAP (REQMOD stream)

The option of sending only HTTP requests to an external ICAP service can be used when integrating with external DLP systems (for example, Infowatch Traffic Monitor).

1. Transferring all HTTP requests to an external ICAP service
   
   icap_service is_ext_req reqmod_precache bypass=0 icap://IPADDRESS:PORT/REQMODPATH
adaptation_service_chain is_req_chain is_ext_req is_kav_req
adaptation_access is_req_chain allow all
    
2. Transfering HTTP requests to an external ICAP service with POST and PUT methods only
   
   icap_service is_ext_req reqmod_precache bypass=0 icap://IPADDRESS:PORT/REQMODPATH
adaptation_service_chain is_req_chain is_ext_req is_kav_req
acl acl_inspect_methods method POST PUT
adaptation_access is_req_chain deny !acl_inspect_methods
adaptation_access is_req_chain allow all
    
3. Similar to previous point + additional filter - do not send requests from certain accounts (username starts with svc_)
   
   icap_service is_ext_req reqmod_precache bypass=0 icap://IPADDRESS:PORT/REQMODPATH
adaptation_service_chain is_req_chain is_ext_req is_kav_req
acl acl_inspect_methods method POST PUT
adaptation_access is_req_chain deny !acl_inspect_methods
acl acl_bypass_users proxy_auth_regex -i svc_.*
adaptation_access is_req_chain deny acl_bypass_users
adaptation_access is_req_chain allow all
    
4. Similar to option 1 point 2 + additional filter - do not send requests when accessing certain URLs from the /etc/squid/bypass_urls.txt file
   
   icap_service is_ext_req reqmod_precache bypass=0 icap://IPADDRESS:PORT/REQMODPATH
adaptation_service_chain is_req_chain is_ext_req is_kav_req
acl acl_inspect_methods method POST PUT
adaptation_access is_req_chain deny !acl_inspect_methods
acl acl_bypass_urls url_regex "/etc/squid/bypass_urls.txt"
adaptation_access is_req_chain deny acl_bypass_urls
adaptation_access is_req_chain allow all

Option 2: Send only HTTP responses to external ICAP (RESPMOD stream)

The option of sending only HTTP requests to an external ICAP service can be used when integrating with external incoming traffic analysis systems, such as Kaspersky Anti Targeted Attack Platform.

1. Sending all HTTP responses to an external ICAP service
   
   icap_service is_ext_resp respmod_precache bypass=0 icap://IPADDRESS:PORT/RESPMODPATH
adaptation_service_chain is_resp_chain is_ext_resp is_kav_resp
adaptation_access is_resp_chain allow all
    
2. Similar to previous point + additional filter - do not send requests from certain accounts (username starts with svc_)
   
   icap_service is_ext_resp respmod_precache bypass=0 icap://IPADDRESS:PORT/RESPMODPATH
adaptation_service_chain is_resp_chain is_ext_resp is_kav_resp
acl acl_bypass_users proxy_auth_regex -i svc_.*
adaptation_access is_resp_chain deny acl_bypass_users
adaptation_access is_resp_chain allow all
3. Similar to option 2 point 1 + additional filter - do not send requests when accessing certain URLs from the /etc/squid/bypass_urls.txt file
   
   icap_service is_ext_resp respmod_precache bypass=0 icap://IPADDRESS:PORT/RESPMODPATH
adaptation_service_chain is_resp_chain is_ext_resp is_kav_resp
acl acl_bypass_urls url_regex "/etc/squid/bypass_urls.txt"
adaptation_access is_resp_chain deny acl_bypass_urls
adaptation_access is_resp_chain allow all

Option 3. Sending both HTTP requests and HTTP responses to external ICAP (REQMOD and RESPMOD streams)

The option of sending HTTP requests/responses to an external ICAP can be used when integrating with external web traffic analysis systems that require both data streams, or when combining two external services according to options 1 and 2.

1. Transferring all HTTP requests/responses to an external ICAP service
   
   icap_service is_ext_req reqmod_precache bypass=0 icap://IPADDRESS:PORT/REQMODPATH
icap_service is_ext_resp respmod_precache bypass=0 icap://IPADDRESS:PORT/RESPMODPATH
adaptation_service_chain is_req_chain is_ext_req is_kav_req
adaptation_service_chain is_resp_chain is_ext_resp is_kav_resp
adaptation_access is_req_chain allow all
adaptation_access is_resp_chain allow all
    
2. Similar to previous point + additional filter - do not send requests from certain accounts (username starts with svc_)
   
   icap_service is_ext_req reqmod_precache bypass=0 icap://IPADDRESS:PORT/REQMODPATH
icap_service is_ext_resp respmod_precache bypass=0 icap://IPADDRESS:PORT/RESPMODPATH
adaptation_service_chain is_req_chain is_ext_req is_kav_req
adaptation_service_chain is_resp_chain is_ext_resp is_kav_resp
acl acl_bypass_users proxy_auth_regex -i svc_.*
adaptation_access is_req_chain deny acl_bypass_users
adaptation_access is_resp_chain deny acl_bypass_users
adaptation_access is_req_chain allow all
adaptation_access is_resp_chain allow all
    
3. Similar to option 3 point 1 + additional filter - do not send requests when accessing certain URLs from the /etc/squid/bypass_urls.txt file
   
   icap_service is_ext_req reqmod_precache bypass=0 icap://IPADDRESS:PORT/REQMODPATH
icap_service is_ext_resp respmod_precache bypass=0 icap://IPADDRESS:PORT/RESPMODPATH
adaptation_service_chain is_req_chain is_ext_req is_kav_req
adaptation_service_chain is_resp_chain is_ext_resp is_kav_resp
acl acl_bypass_urls url_regex "/etc/squid/bypass_urls.txt"
adaptation_access is_req_chain deny acl_bypass_urls
adaptation_access is_resp_chain deny acl_bypass_urls
adaptation_access is_req_chain allow all
adaptation_access is_resp_chain allow all

Making changes to the built-in proxy server configuration

The option of sending only HTTP requests to an external ICAP service can be used when integrating with external incoming traffic analysis systems, such as Kaspersky Anti Targeted Attack Platform.

1. Connect to the cluster node via SSH to access the technical support mode.
    
2. If the selected configuration option requires an external file with access lists (for example, bypass_urls.txt for options 1.4, 2.3, 3.3), place it in the /etc/squid directory. This must be done before any changes are made to the built-in proxy configuration template.
    
3. Change to the directory where the built-in proxy configuration file templates are located:
   
   cd /opt/kaspersky/kwts-appliance-addon/share/templates
    
4. Make a backup copy of the squid.conf.template file if you haven't already:
   
   cp -p squid.conf.template squid.conf.template.backup
    
5. Open the squid.conf.template file for editing using a text editor:
   
   vim squid.conf.template
6. Go to the end of the file, paste the configuration fragment for integration with an external ICAP service in the place indicated below (existing lines are marked in black, they do not need to be modified in any way, green is the lines to be added)
   
   adaptation_send_client_ip on
adaptation_send_username on
icap_enable on
icap_service is_kav_req reqmod_precache icap://127.0.0.1:1344/av/reqmod
icap_service is_kav_resp respmod_precache icap://127.0.0.1:1344/av/respmod
### --> put your external ICAP configuration here <-- ###
adaptation_access is_kav_req allow all
adaptation_access is_kav_resp allow all
icap_service_failure_limit -1
   
   An example of inserting a configuration fragment (for option 1.2):
   
   adaptation_send_client_ip on
adaptation_send_username on
icap_enable on
icap_service is_kav_req reqmod_precache icap://127.0.0.1:1344/av/reqmod
icap_service is_kav_resp respmod_precache icap://127.0.0.1:1344/av/respmod
### External ICAP configuration begin ###
icap_service is_ext_req reqmod_precache bypass=0 icap://x.x.x.x/reqmod
adaptation_service_chain is_req_chain is_ext_req is_kav_req
acl acl_inspect_methods method POST PUT
adaptation_access is_req_chain deny !acl_inspect_methods
adaptation_access is_req_chain allow all
### External ICAP configuration end ###
adaptation_access is_kav_req allow all
adaptation_access is_kav_resp allow all
icap_service_failure_limit -1
7. Save changes to squid.conf.template
    
8. In order for the changes in the template to be applied, change some setting of the built-in proxy server through the web interface. For example, you can turn off logging (Settings - Built-in proxy server - Log), save the changes, and then return the previous value back.
    
9. Check that the changes have made their way into the main configuration file of the built-in proxy server:
   less /etc/squid/squid.conf
    
10. Check the status of the squid service, it should be running:
    systemctl status squid

This completes the procedure. The described actions must be repeated on each node of the Kaspersky Web Traffic Security cluster.