Any connection between KART 5 (3660) and BSOD Critical Service Failed

[steve_paul_quinn]

Great work CF93.  Thanks for sharing your details.  Hopefully it will help others who land here and find this thread.  😎

 

@pdwkand you have done a great work on this topic and it has helped me a lot too, thank you.

 

For my tests, regarding the second setup of KART on my VM, despite the KART update notice, Windows log says that there was no update and KART is not updating yet.

 

 

Thanks.  It feels good to help others save hair.  :-)

Regarding your tests, it sounds like Kaspersky may have wisely paused the problematic update.

 

 

[steve_paul_quinn]

Hi Folks.  I noticed 3.0.1.3886 was pushed out and upgraded in the last 24 hours.  I’ve rebooted, all is ok and continue to test.

Notes here

https://docs.google.com/document/d/1gDzDaWPk4s8L2eqP6qERVojFXUtevCaca2G7SmO0wK8/edit

 

 

[CF93]

Hi Folks.  I noticed 3.0.1.3886 was pushed out and upgraded in the last 24 hours.  I’ve rebooted, all is ok and continue to test.

Notes here

https://docs.google.com/document/d/1gDzDaWPk4s8L2eqP6qERVojFXUtevCaca2G7SmO0wK8/edit

 

 

 

Hi, this is a good news but Kaspersky is very late on solving this bad problem.

I wonder how many people have been impacted in the world before a solution...

 

 

[Marc5]

 

Hi, this is a good news but Kaspersky is very late on solving thbad problem.

I wonder how many people have been impacted in the world before a solution...

 

We will never touch another Kaspersky product after this fiasco.
 

[Vasily Burov]

Hi, All!
Sorry for late response. Thank you for your patience and for reporting about these issues. The problem with UpperFilters registry value was fixed in last product update. We continue investigation of BSOD problem but we think that this may be due to the first problem with the registry.

P.S. The actual version of the product can be found here: 
Menu -> Get Support

Thanks.

 

[steve_paul_quinn]

Hi, All!
Sorry for late response. Thank you for your patience and for reporting about these issues. The problem with UpperFilters registry value was fixed in last product update. We continue investigation of BSOD problem but we think that this may be due to the first problem with the registry.

P.S. The actual version of the product can be found here: 
Menu -> Get Support

Thanks.

 

 

Hi Vasily

 

I really appreciate an official Kaspersky representative reaching out.  I’m sure you would agree the response time has been very slow.  I hope that whatever the cause of the slow response has been addressed.  I have many customers who are now afraid of Kaspersky software.  You may also notice there are others who voiced these same opinions in this forum thread.  I ‘d like to work together to gain back their trust.

 

As you may see from my contributions in this thread, I have a HP ZBook G1 that had experienced the BSOD issue.  I reverted to a previous backup and have monitored the machine for many days.  I am hoping to assist with catching the BSOD bug.  As expected, the machine automatically upgraded to 5.0.0.3886(i).  I have enabled Event Logging with Maximum detail.  I’ve also created a reboot task for every 30 minutes.  It has rebooted without issue for 3 days.

 

If there are any changes or suggestions to my test environment, I would appreciate your input

 

Take care

 

Steve Quinn

 

 

[Vasily Burov]

 

Hi Vasily

 

I really appreciate an official Kaspersky representative reaching out.  I’m sure you would agree the response time has been very slow.  I hope that whatever the cause of the slow response has been addressed.  I have many customers who are now afraid of Kaspersky software.  You may also notice there are others who voiced these same opinions in this forum thread.  I ‘d like to work together to gain back their trust.

 

As you may see from my contributions in this thread, I have a HP ZBook G1 that had experienced the BSOD issue.  I reverted to a previous backup and have monitored the machine for many days.  I am hoping to assist with catching the BSOD bug.  As expected, the machine automatically upgraded to 5.0.0.3886(i).  I have enabled Event Logging with Maximum detail.  I’ve also created a reboot task for every 30 minutes.  It has rebooted without issue for 3 days.

 

If there are any changes or suggestions to my test environment, I would appreciate your input

 

Take care

 

Steve Quinn

 

 

Hi, Steve!

Thanks for you reply. I agree that response time was long and hope that this situation will not repeats again. 

In our lab we can not reproduce this BSOD at the moment. I need some time to think about how we can reproduce this on your configuration, I will be discuss that with colleagues tomorrow. 
I really want to catch this BSOD. 😡

Thanks.

[steve_paul_quinn]

 

Hi Vasily

 

I really appreciate an official Kaspersky representative reaching out.  I’m sure you would agree the response time has been very slow.  I hope that whatever the cause of the slow response has been addressed.  I have many customers who are now afraid of Kaspersky software.  You may also notice there are others who voiced these same opinions in this forum thread.  I ‘d like to work together to gain back their trust.

 

As you may see from my contributions in this thread, I have a HP ZBook G1 that had experienced the BSOD issue.  I reverted to a previous backup and have monitored the machine for many days.  I am hoping to assist with catching the BSOD bug.  As expected, the machine automatically upgraded to 5.0.0.3886(i).  I have enabled Event Logging with Maximum detail.  I’ve also created a reboot task for every 30 minutes.  It has rebooted without issue for 3 days.

 

If there are any changes or suggestions to my test environment, I would appreciate your input

 

Take care

 

Steve Quinn

 

 

Hi, Steve!

Thanks for you reply. I agree that response time was long and hope that this situation will not repeats again. 

In our lab we can not reproduce this BSOD at the moment. I need some time to think about how we can reproduce this on your configuration, I will be discuss that with colleagues tomorrow. 
I really want to catch this BSOD. 😡

Thanks.

 

Hi Vasily

You are very welcome.  I too want to catch it.  It’s a very nasty bug.

Many of us noticed that the BSOD occurred with 3660 a few days after UpperFilters was removed.  I am very tempted to restore my test machine to 3660, disable the network interface to prevent an upgrade and see if I can recreate the problem.  Do you think this would be helpful?

Steve

 

 

[steve_paul_quinn]

 

Hi Vasily

 

I really appreciate an official Kaspersky representative reaching out.  I’m sure you would agree the response time has been very slow.  I hope that whatever the cause of the slow response has been addressed.  I have many customers who are now afraid of Kaspersky software.  You may also notice there are others who voiced these same opinions in this forum thread.  I ‘d like to work together to gain back their trust.

 

As you may see from my contributions in this thread, I have a HP ZBook G1 that had experienced the BSOD issue.  I reverted to a previous backup and have monitored the machine for many days.  I am hoping to assist with catching the BSOD bug.  As expected, the machine automatically upgraded to 5.0.0.3886(i).  I have enabled Event Logging with Maximum detail.  I’ve also created a reboot task for every 30 minutes.  It has rebooted without issue for 3 days.

 

If there are any changes or suggestions to my test environment, I would appreciate your input

 

Take care

 

Steve Quinn

 

 

Hi, Steve!

Thanks for you reply. I agree that response time was long and hope that this situation will not repeats again. 

In our lab we can not reproduce this BSOD at the moment. I need some time to think about how we can reproduce this on your configuration, I will be discuss that with colleagues tomorrow. 
I really want to catch this BSOD. 😡

Thanks.

 

 

Hi Vasily

I have a parallel test idea to keep momentum on troubleshooting this issue since we may be in different time zones.  I keep a small repo of files at home for offline use.

I noticed I have KART_5.0.0.92320-Home.exe  This installer is Product version 3.0.1.3660 which we have seen be a problem.

I have a second ZBook G2 I use for testing stuff with VMware Workstation

I'll spin up a similar vintage Windows 10 x64 Pro VM to test out 3660
I'll try to block KART or disable networking so as to keep it at 3660

FYI the Zbook G1 physical testing and now virtual testing will be done with Windows 10 x64 Pro 1909

If you have adjustments or suggestions I'm all ears

Take care

Steve

[lnet]

Hi, All!
Sorry for late response. Thank you for your patience and for reporting about these issues. The problem with UpperFilters registry value was fixed in last product update. We continue investigation of BSOD problem but we think that this may be due to the first problem with the registry.

P.S. The actual version of the product can be found here: 
Menu -> Get Support

Thanks.

 

Hello, can you please tell us how we can make sure if a computer is affected and will start as soon we restart it ?

We have many Servers (SBS 2011, Windows 2008R2, Windows 2016 Server, Windows 2019 Server) which have never been restarted since the problem arised.

Please do respond soon to this urgent issue !

 

If somebody else has a definitive answer to this question i am happy to hear about.

 

Regards,

 

Mike

[Vasily Burov]

Hello, can you please tell us how we can make sure if a computer is affected and will start as soon we restart it ?

We have many Servers (SBS 2011, Windows 2008R2, Windows 2016 Server, Windows 2019 Server) which have never been restarted since the problem arised.

Please do respond soon to this urgent issue !

 

If somebody else has a definitive answer to this question i am happy to hear about.

 

Regards,

 

Mike

 

Hi, Mike!

I don’t know what computers will be affected by this issue. In our labs we don’t have the same problem :-( We try to reproduce it. At this moment we think that fix of the UpperFilters registry value in last product update will fix the BSOD too.

Can anybody to tell about the sequence of updates install that led to BSOD:
Windows KB500802/KB500808 update was installed, then update for KART and then computer was restarted
or
KART update was installed, then Windows KB500802/KB500808 update and then computer was restarted
or
KART update was installed and then computer was restarted but windows KB500802/KB500808 update installation was still in progress
or
It does not matter?

I appreciate you for help.

[Vasily Burov]

Hi Vasily

You are very welcome.  I too want to catch it.  It’s a very nasty bug.

Many of us noticed that the BSOD occurred with 3660 a few days after UpperFilters was removed.  I am very tempted to restore my test machine to 3660, disable the network interface to prevent an upgrade and see if I can recreate the problem.  Do you think this would be helpful?

Steve

 

Hi, Steve!

It will be very helpful! Please try to reproduce this BSOD with product logs on maximum level. 

Can you list here the value of UpperFilters registry parameter on restored machine before product update?

Thanks.

[steve_paul_quinn]

Hi Vasily

You are very welcome.  I too want to catch it.  It’s a very nasty bug.

Many of us noticed that the BSOD occurred with 3660 a few days after UpperFilters was removed.  I am very tempted to restore my test machine to 3660, disable the network interface to prevent an upgrade and see if I can recreate the problem.  Do you think this would be helpful?

Steve

 

Hi, Steve!

It will be very helpful! Please try to reproduce this BSOD with product logs on maximum level. 

Can you list here the value of UpperFilters registry parameter on restored machine before product update?

Thanks.

 

Hi Vasily

Sure.  Here you go.  UpperFilters has a single Data entry of volsnap

 

 

 

[steve_paul_quinn]

Hello, can you please tell us how we can make sure if a computer is affected and will start as soon we restart it ?

We have many Servers (SBS 2011, Windows 2008R2, Windows 2016 Server, Windows 2019 Server) which have never been restarted since the problem arised.

Please do respond soon to this urgent issue !

 

If somebody else has a definitive answer to this question i am happy to hear about.

 

Regards,

 

Mike

 

Hi, Mike!

I don’t know what computers will be affected by this issue. In our labs we don’t have the same problem :-( We try to reproduce it. At this moment we think that fix of the UpperFilters registry value in last product update will fix the BSOD too.

Can anybody to tell about the sequence of updates install that led to BSOD:
Windows KB500802/KB500808 update was installed, then update for KART and then computer was restarted
or
KART update was installed, then Windows KB500802/KB500808 update and then computer was restarted
or
KART update was installed and then computer was restarted but windows KB500802/KB500808 update installation was still in progress
or
It does not matter?

I appreciate you for help.

 

 

Hi Vasily

I created for reference, a Macrium backup of my physical Zbook after I experienced the BSOD.  I will recover it and look at the status of KB500802/KB500808.  I have honestly been shell shocked by poor MS patch stability for quite some time now.  There is a possibility that I had disabled Windows Updates using Windows Update Blocker. https://www.sordum.org/9470/windows-update-blocker-v1-6/  I will confirm for you.

Hope this helps

Steve

[steve_paul_quinn]

Hi Vasily

It is confirmed.  Windows Update Blocker v1.5 was used and Windows updates is disabled.  In my situation the BSOD issue was independent of KB500802/KB500808.  I’ve included the KB that were installed and the Windows Version as 1909

 

 

 

 

 

 

[steve_paul_quinn]

Hi Vasily

I have an idea to accelerate your research.

I have a Macrium backup of my Zbook after the BSOD occurred.  In restoring this image to investigate I have to do the following process

1 Restore the Macrium Image

2 Restore the CatRoot and DriverStore files

3 Manually recursively delete C:\Kaspersky Lab 

If I skip step 3, CatRoot and DriverStore are deleted on the next reboot.

Would a copy of this Macrium image not be helpful for your team to investigate?  I can somehow upload it to you for a Physical restore or VM.   It wont reveal the triggers to cause the issue but it may be helpful for a post mortem analysis. Just an idea :-)

Take care

Steve

 

 

 

 

 

[Vasily Burov]

Hi Vasily

It is confirmed.  Windows Update Blocker v1.5 was used and Windows updates is disabled.  In my situation the BSOD issue was independent of KB500802/KB500808.  I’ve included the KB that were installed and the Windows Version as 1909

 

 

 

 

 

 

Hi, Steve

Thanks for very useful info! 

KART update was installed and popup asking to restart is displayed - when you restarted the PC after that (immediately or with delay)? Can you write the sequence of your actions?  

[Vasily Burov]

 

3 Manually recursively delete C:\Kaspersky Lab 

 

Did you installed the product in non-default path?

[steve_paul_quinn]

Hi Vasily

It is confirmed.  Windows Update Blocker v1.5 was used and Windows updates is disabled.  In my situation the BSOD issue was independent of KB500802/KB500808.  I’ve included the KB that were installed and the Windows Version as 1909

 

 

 

 

 

 

Hi, Steve

Thanks for very useful info! 

KART update was installed and popup asking to restart is displayed - when you restarted the PC after that (immediately or with delay)? Can you write the sequence of your actions?  

 

Hi Vasily

I’m sorry but I cannot recall the exact sequence of events prior to the BSOD as the issue occurred on March 28 2021.  I do recall turning on my Zbook while I was working with a customer experiencing the same issue.  I’m pretty sure I was prompted to restart for a KART update and I did restart immediately.  I hope this helps.

Steve

[steve_paul_quinn]

 

3 Manually recursively delete C:\Kaspersky Lab 

 

Did you installed the product in non-default path?

 

Oops my bad sorry. 

I manually recursively deleted C:\Program Files (x86)\Kaspersky Lab\

[steve_paul_quinn]

Hi, All!
Sorry for late response. Thank you for your patience and for reporting about these issues. The problem with UpperFilters registry value was fixed in last product update. We continue investigation of BSOD problem but we think that this may be due to the first problem with the registry.

P.S. The actual version of the product can be found here: 
Menu -> Get Support

Thanks.

 

Hello, can you please tell us how we can make sure if a computer is affected and will start as soon we restart it ?

We have many Servers (SBS 2011, Windows 2008R2, Windows 2016 Server, Windows 2019 Server) which have never been restarted since the problem arised.

Please do respond soon to this urgent issue !

 

If somebody else has a definitive answer to this question i am happy to hear about.

 

Regards,

 

Mike

 

 

Hi Mike

This forum thread is getting kinda messy and hard to follow.  Hopefully I can help you.  I’ll summarize if you have not reviewed this entire thread.  I’ve got a laptop with a Macrium backup of the BSOD issue on Windows 10 x64 1909.  Perhaps what I know will help you with your Windows Servers.

The first clue of a KART issue is the removal of the UpperFilters registry entry.  This is easy to check for.   Several days later, it appears a BSOD is caused by the removal of CatRoot and DriverStore during the next reboot.  I know of no way predict this.  It did happen for many of us with KART Application version 3660 which has been since upgraded.  Time will tell if the BSOD risk remains.

For all my customers, I am removing KART from their “working” machines to prevent the BSOD risk.  If the KART files are not present during the next reboot, CatRoot and DriverStore “should” remain intact.

If I was you, I would create a small repo of CatRoot and DriverStore files for all your system variants from working machines.  Just in case they are needed.  I would also prepare and test a working PE recovery environment proactively so recovery is not in a panic when needed.  I like Macrium for this and there are certainly others to choose from.

Hope this helps

Steve

 

 

 

I hope this helps

 

 

 

 

[steve_paul_quinn]

Hi Vasily/Kaspersky Team

Perhaps my original post got lost in this thread.  I will repost and await a reply

 

I have an idea to accelerate your research.

I have a Macrium backup of my Zbook after the BSOD occurred.  In restoring this image to investigate I have to do the following process

1 Restore the Macrium Image

2 Restore the CatRoot and DriverStore files

3 Manually recursively delete C:\Program Files (x86)\Kaspersky Lab\

If I skip step 3, CatRoot and DriverStore are deleted on the next reboot.

Would a copy of this Macrium image not be helpful for your team to investigate?  I can somehow upload it to you for a Physical restore or VM.   It wont reveal the triggers to cause the issue but it may be helpful for a post mortem analysis. Just an idea :-)

Take care

Steve

[steve_paul_quinn]

Hi Vasily/Kaspersky Team

It’s been several days with no official response from Kaspersky.

Can you please give us an update?

Steve

 

 

[Vasily Burov]

Hi Vasily/Kaspersky Team

It’s been several days with no official response from Kaspersky.

Can you please give us an update?

Steve

 

Hi, Steve!

Sorry for delay, in Russia we have small holidays :-) We still can’t reproduce this issue :-( Thank you for submitting the playback algorithm. I suggest the following way:

1. Restore the Macrium Image
2. Enable product logging on maximum level.
3. Restart the product
4. Restore the CatRoot and DriverStore files
5. Restart the computer and get the BSOD
6. After memory dump is created - restart computer again in safe mode
7. Save product logs (please see “Log application events” chapter in online help) and memory dump to another location.

Please write here the message if you succeed. I will consult with our legal department about the method of transferring traces to us.

Thanks!

 

[steve_paul_quinn]

Hi Vasily/Kaspersky Team

It’s been several days with no official response from Kaspersky.

Can you please give us an update?

Steve

 

Hi, Steve!

Sorry for delay, in Russia we have small holidays :-) We still can’t reproduce this issue :-( Thank you for submitting the playback algorithm. I suggest the following way:

1. Restore the Macrium Image
2. Enable product logging on maximum level.
3. Restart the product
4. Restore the CatRoot and DriverStore files
5. Restart the computer and get the BSOD
6. After memory dump is created - restart computer again in safe mode
7. Save product logs (please see “Log application events” chapter in online help) and memory dump to another location.

Please write here the message if you succeed. I will consult with our legal department about the method of transferring traces to us.

Thanks!

 

 

Hi Folks

Thanks for the update.  I hope you had a nice holiday.

I only have a Macrium image of the machine AFTER a BSOD. 

I will need to adjust the process slightly.  I will try this tomorrow, it was a long day.

 

1 Restore the BSOD Macrium Image

2 Restore the CatRoot and DriverStore files

3 Rename the Kaspersky Lab files so they do not delete CatRoot and DriveStore on the next reboot

4 Restart the machine

5 Somehow enable product logging on maximum level.  I hope I can.

6 Rename the Kaspersky Lab files back to their original names

7 Reboot, hope for a BSOD and get you the logs

 

Any adjustments or suggestions?