Kaspersky is breaking ApolloClient requests(GraphQL)

[nitzansi]

Not sure this is the right forum, please redirect to the relevant one if there is one.
We are getting complaints from our users that their site is not working.
Looking into it, we found out that the exception is being thrown from Kaspersky code, a file called `main.js`, in a function called `copyHeadersValue`

this is the stack trace:

"TypeError: value.toString is not a function\n at CopyHeadersValue (https://gc.kis.v2.scr.kaspersky-labs.com/FD126C42-EBFA-4E12-B309-BB3FDD723AC1/main.js?attr=fVzR6GWpTiGSWVNM_NZXCRQJPJp1YS83KyHeXrNEYHrQcGytgDBRQf_T4qtafYO_Naziog3E0EyHZxlIGcm0MLu9Fr9KBOeT6jziCBmpYyVCYk2_9sSSSNRBFLcbAlfU-1Cemzd9G8W-5q0a1FQt93BzjfgeI9I3JQLlMpPW7Af68xjJ8PD9XB4naOIJ2EXPuQ2Cq8YQIlPndkWTkJNbeQ:3742:15)\n at fetchCallImpl (https://gc.kis.v2.scr.kaspersky-labs.com/FD126C42-EBFA-4E12-B309-BB3FDD723AC1/main.js?attr=fVzR6GWpTiGSWVNM_NZXCRQJPJp1YS83KyHeXrNEYHrQcGytgDBRQf_T4qtafYO_Naziog3E0EyHZxlIGcm0MLu9Fr9KBOeT6jziCBmpYyVCYk2_9sSSSNRBFLcbAlfU-1Cemzd9G8W-5q0a1FQt93BzjfgeI9I3JQLlMpPW7Af68xjJ8PD9XB4naOIJ2EXPuQ2Cq8YQIlPndkWTkJNbeQ:3783:25)\n at fetch (https://gc.kis.v2.scr.kaspersky-labs.com/FD126C42-EBFA-4E12-B309-BB3FDD723AC1/main.js?attr=fVzR6GWpTiGSWVNM_NZXCRQJPJp1YS83KyHeXrNEYHrQcGytgDBRQf_T4qtafYO_Naziog3E0EyHZxlIGcm0MLu9Fr9KBOeT6jziCBmpYyVCYk2_9sSSSNRBFLcbAlfU-1Cemzd9G8W-5q0a1FQt93BzjfgeI9I3JQLlMpPW7Af68xjJ8PD9XB4naOIJ2EXPuQ2Cq8YQIlPndkWTkJNbeQ:3810:60)\n at https://static.parastorage.com/unpkg/@sentry/browser@5.30.0/build/bundle.min.js:2:14963\n

main.js is attached(without the signature): https : //file.io/hxWvagpw3sjZ

We checked it only with Kaspersky Free, but I guess it happens in more versions.

Edited December 8 by Berny

[Berny]

@nitzansi Welcome.

Please temporary disable ' Encrypted connections scan ' ?

Quote

" Scanning of encrypted connections is required to run the following protection components: Mail Anti-Virus, Safe Money, URL Advisor, Private Browsing, Safe Browsing, and Anti-Banner "

 

↓ " main.js is attached " ↓

Spoiler

 

[MA Cosgrove]

We are having the same problem in our app. Lots of support requests coming in.

[MA Cosgrove]

@berny My support team is asking for instructions for users on how to change that Encrypted Connections Scan setting. I can't do that as I don't have Kaspersky myself. The page you linked to only described the setting without showing how to change it.  Can you provide some detailed instructions please.

[MA Cosgrove]

My colleague has suggested:

Rather than disabling it globally, it sounds like you might be able to disable it for certain domains. Perhaps the users could contact Kaspersky directly or their orgs IT team for advice.

"If you do not want the application to verify an SSL connection with a website, you can add the website to the list of exclusions by clicking the Configure trusted addresses link."

Is this likely to work?

[Flood and Flood's wife]

My support team is asking for instructions for users on how to change that Encrypted Connections Scan setting. I can't do that as I don't have Kaspersky myself. The page you linked to only described the setting without showing how to change it.  Can you provide some detailed instructions please.

My colleague has suggested:

Rather than disabling it globally, it sounds like you might be able to disable it for certain domains. Perhaps the users could contact Kaspersky directly or their orgs IT team for advice.

"If you do not want the application to verify an SSL connection with a website, you can add the website to the list of exclusions by clicking the Configure trusted addresses link."

Is this likely to work?

Hello @MA Cosgrove, 

Thank you for posting back & the additional information!

To follow the suggestion by @Berny, select Security Settings, select Network Settings, *uncheck* Inject script into web traffic to interact with web pages, select Continue, select Save, select Confirm. Note: This will impact the correct operation of such components as Safe Money, Private Browsing, Anti-Banner, and URL Advisor & should be seen as a *test* only. 

To add an exclusion, select Security Settings, select Network Settings, select Configure trusted addresses, select Add - add the address/domain, select Add (again), select Save - READ: Allow and add domain to exclusions

*IF* issues persist after adding exclusion/s please log a request with Kaspersky Customer Service, https://support.kaspersky.com/b2c#contacts  - on the support page, select either Chat or Email, then fill in the template as shown; select the Kaspersky application from the drop-down-list (A)

*Also* -> IF using Chat - ask the operator to email (you) a copy of the chat transcript *before* ending the chat - otherwise (you'll) have no record of the chat*

Thank you🙏
Flood🐳+🐋

Edited December 9 by Flood and Flood's wife
added images

[nitzansi]

Hey @Berny, I'm coming from a large company serving milions of users, it is really hard for them to identify the error is coming from kaspersky. This error started popping up around Nov 14th and I believe your dev team can apply a simple fix to work around it. This is a plain Javascript error which can easily be avoided.
How can I contact your dev team around that area of the application? 
Thanks 

[phryneas]

While this is definitely a Kaspersky bug, we have taken precautions on the Apollo Client side to prevent this kind of buggy-Antivirus-caused problem a few months ago.

If you can (I know, it's a big ask to spontaneously do this to a production app), please try to update Apollo Client to at least version 3.11.8. (And please report back if it works if you do)

 

At Kaspersky: the fact that this can be patched doesn't take the pressure off of you - there are still hundreds of thousands of webpages using older versions of Apollo Client.
npm downloads for those old versions are > three million per week, and every one of those downloads can be built into a website. Please get this fixed ASAP.

Edited December 9 by phryneas

[Berny]

Hi @nitzansi @phryneas

2 hours ago, nitzansi said:

How can I contact your dev team around that area of the application?

 

9 minutes ago, phryneas said:

Please get this fixed ASAP.

This is a user Forum , please contact Kaspersky Technical Support :
→ https://support.kaspersky.com/b2c/#contacts

[harlan4096]

Additionally, if there is an Apollo Client exe or any kind of service involved, You can try to add an exclusion via Intrusion Prevention -> Manage Applications, search and find Apollo Client exe / services involved, edit their rules, and add an exclusion for encrypted connections. An example:

 

 

Just found, double click over the exe / services, then go to tab Exclusions, enable the check button as showed in the pic, and Save the rule, reboot the system and try again.

[phryneas]

@Bernymost of us are not customers of Kaspersky, but engineers of the companies whose websites you are currently breaking, so I'm not sure if we can even get in on customer service.

Please escalate this internally; this has a bigger impact than you assume right now.

Here is a list of Websites this affects: https : //trends.builtwith.com/websitelist/Apollo-GraphQL

 

@harlan4096 This is not affecting an executable, it's affecting hundreds of thousands of websites. The page I linked to above says something along the lines of 1.3 million websites and is likely incomplete.

Edited December 9 by Berny

[Igor Kurzin]

@phryneas, can you please let me know an example of a website where  I can reproduce this problem?Namely, what are the steps to reproduce the error? Reproduction in-lab allows us to analyze and find fix quicker. 

@nitzansi, the issue occurs at wix.com? Can you tell me please the steps to reproduce on my end? 

@MA Cosgrove, same question, please provide instructions how to reproduce, if possible. 

Thank you. 

[phryneas]

Here is one of our CI builds that should still use an affected version: (I'm still setting up a VM to reproduce this myself, so I can't point to external websites - I don't know which version they are using)

https : //apollo-git-8b780d-apollo-client-next-package-integration-tests.vercel.app/cc/dynamic/useSuspenseQueryWithError

 

The bug: your `CopyHeadersValue` function is trying to call `headers.toString()`. `headers` in question is an object created via `Object.create(null)`, so it has no prototype and no `toString` function as a result. You have to call `Object.prototype.toString.call(headers)` instead.

Here is a piece of code reproducing the bug and the solution:

 

{
  const headers = Object.create(null)
  try {
    const value = headers.toString()
    console.log(value)
  } catch (e) {
    console.error("accessing `headers.toString()` does not work: ", e)
  }
  const value = Object.prototype.toString.call(headers)
  console.log("this worked: ", value)
}

 

@Igor Kurzin forgot to tag you - please see the message above

Edited December 9 by phryneas

[Igor Kurzin]

@phryneas, thanks for the information, we are checking. Will keep you updated. 

[nitzansi]

@phryneas I'm working on updating our apollo client version, will update ASAP
@Igor Kurzin please provide me your email and I'll be able to create the scenario for you, it is a bit complicated to get to the specific failing flow.

[Igor Kurzin]

@nitzansi, pm'ed the email, thanks much! 

[MA Cosgrove]

20 hours ago, Igor Kurzin said:

@phryneas, can you please let me know an example of a website where  I can reproduce this problem?Namely, what are the steps to reproduce the error? Reproduction in-lab allows us to analyze and find fix quicker. 

@nitzansi, the issue occurs at wix.com? Can you tell me please the steps to reproduce on my end? 

@MA Cosgrove, same question, please provide instructions how to reproduce, if possible. 

Thank you. 

Create a free trial account at https://app.covidence.org/ on a machine with Kaspersky installed, and try to create a new review.

[Flood and Flood's wife]

7 hours ago, MA Cosgrove said:

Create a free trial account at https://app.covidence.org/ on a machine with Kaspersky installed, and try to create a new review.

Hello @MA Cosgrove & @Igor Kurzin

Thank you🙏
Flood🐳+🐋

[Igor Kurzin]

@MA Cosgrove, what is the expected and actual result? In my test nothing happens with I click "Create review" button (see attached screenshot). Is this the issue? 

[phryneas]

Yesterday evening I finally managed to set up a VM that would actually run Kaspersky, visited the page I linked above and immediately got this:

Unfortunately, at that point I called it quits and this morning, I seem I can't reproduce the problem anymore.
I'm still investigating, it might be a cache thing that prevents it from happening a second time.

Seems to be a cache thing, but I can reproduce it on a random website:

Go e.g. to example.com and execute this in the devtools:

    const headers = Object.create(null)
    headers.foo = "bar";
    fetch("/", {headers})

 

@Igor Kurzin the error is cause by https://me.kis.v2.scr.kaspersky-labs.com/FD126C42-EBFA-4E12-B309-BB3FDD723AC1/main.js?attr=TTtVSzJmXF0BU9K9voTozInYCUlc64TEB1JqU8bGAhRBcBEl_QdI75iTvatoPOJu line 3026:

 

[Igor Kurzin]

@phryneas, 

Quote

this morning, I seem I can't reproduce the problem anymore

Try reproducing in browser incognito mode, how will it go? (Via File -> New Incognito Window). 

thanks for the information, passed over to developers. 

[phryneas]

Can confirm, the problem reappears in Edges "InPrivate" mode

[Igor Kurzin]

15 minutes ago, phryneas said:

Can confirm, the problem reappears in Edges "InPrivate" mode

That's what I expected, thank you for checking. 

[phryneas]

@Igor Kurzinis there any update on this?

[Igor Kurzin]

@phryneas, the issue will be fixed for all websites in the next product version 21.20., which is expected end of January/start of February 2025. 

@nitzansithe issue with wix.com has been fixed via a workaround in bases. Customers need to update databases to apply the fix.