Crashes after installing Vulkan SDK

[no-connections]

Describe the Issue
Since you deleted the only option to accelerate on old Nvidia GPU I had to try Vulkan.
This worked once and now crashes Kobold every time. No matter if I select to run CPU only it just crash.

I can see it still try to load something into VRAM and crash When CPU only mode is selected.
So essentially Kobold is bricked by Vulcan with no way to run it any more.

Additional Information:
I tried 1.108.2 as well and it does the same thing.
The model does not seem to matter.

K4000 GPU
Win10

edit

The only version I can run now is old version like 1.101.1 where CLBlast is enabled and selected. Running Vulcan or CPU mode on that version crashes Kobold.

I shuold not need to mention I have been using CPU mode fine up to the point that I installed Vulcan.

[no-connections]

Removing Vulcan did not help. But after a lot of back and forth in the settings, selecting 0 layers and no KV cache (before) selecting CPU seem to workaround the crash. I will have to test if performance is affected or not.

I would love to have CLBlast back or Vulcan actually working but old hardware seem to not be any care for.

[LostRuins]

Did you try the vulkan older pc build? In the oldpc exe

[no-connections]

Vulcan old CPU on the nocuda build is crashing yes. I didn't realize old cpu had an older cpu version of Vulcan but result is the same crash.
I keep trying to see if any Vulcan combination works but it's a slow task due to the old hardware. But it's the only thing I have with enough RAM atm.

And selecting old CPU also crash unless I do the workaround I mentioned by turning off KV cache and setting 0 layers before I switch to CPU only mode. Something I did not have to do before installing Vulcan SDK.
I did try failsafe mode but it just hung with a larger model.

I don't know why or how Vulcan SDK broke everything (vulkansdk-windows-X64-1.4.341.1.exe), or if it even was the right thing to install since google "how to install vulkan" comes up with no useful information so I had to guess.

There seem to be a bug that try to use vulkan even in CPU only mode that cause a crash, and since I didn't have any vulkan API before it was not an issue but now it cause havoc. Older Kobold build seem to be affected as well with version 1.99 tested to also crash like this.

Sorry if I sound a bit scatter brain trying to make sense and throwing tests at the problem until something sticks.
I also don't know if something changed or if it's just in my mind that CLBlast seem more temperamental, or it could be cause I try getting larger models to run. It is mainly BLAS step that is the most painful that really need some sort of acceleration to begin with

[LostRuins]

You don't need to install vulkan sdk to run the vulkan builds. They are self contained. And the pure "use CPU" build will never load vulkan

[henk717]

Is vulkan not a part of the GPU driver? Installing Vulkan SDK doesn't make Vulkan work, installing a driver that has it will.
https://us.download.nvidia.com/Windows/Quadro_Certified/475.14/475.14-quadro-rtx-desktop-notebook-win10-win11-64bit-international-dch-whql.exe is the latest driver for your card.
This should give you Vulkan 1.2 at least on a driver level, and with a bit of luck it also gets you CUDA 11.4 for the oldpc build's cuda.
I don't know if compute level 3.0 is enough (I don't think it is) but if not then Vulkan 1.2 should be.

Update: Did some research and Vulkan 1.1 is part of the ancient 2018 driver you have, so installing the SDK probably added a missing bit that made it be able to use it partially at least. But because your driver is ancient its unstable as I heard it needs to be 1.2 not 1.1. Definitely do that driver update because old drivers aren't supported and with a newer one you have a chance of it working or us being able to request support.

[no-connections]

You don't need to install vulkan sdk to run the vulkan builds. They are self contained. And the pure "use CPU" build will never load vulkan

Well using CPU does load Vulcan and that is why it's crashing. At least from what I can tell. As it works if I turn off KV cache and set layers to 0 before selecting CPU (as those fields disappear then CPU is selected). You can see VRAM is being loaded and cmd prompt mention Vulkan as well. (and to be reeeeely clear, Old CPU is selected in that screenshot).
I'm sure it makes no sense to you since it does not make sense to me but that is what happened.
I installed SDK and CPU mode does not work any more unless I do the workaround I mentioned. (that I of course did not need to do before).

As much as I want to reinstall or mess around the drivers don't want to install for some reason. And I need more space to do a full backup and reinstall or switch to Mint.

To be clear, the issue itself is about CPU mode not working as expected after installing Vulkan SDK, not about getting Vulkan to work.
I don't remember if Kobold auto selected Vulkan before I installed but it does now.

[henk717]

It auto selects Vulkan if it detects a dedicated GPU with Vulkan, but its as simple as selecting Use CPU to switch that back.

[no-connections]

If it was that simple I would not have made this issue. It would have been just one more click and then done. But just selecting CPU does not work any more.
I just added that cause I don't remember if it defaulted to CPU or Vulkan before I installed the SDK.
And I have no idea how to remove Vulkan to test. I did uninstall it but that did not help. I removed the dll in system32 but that did not help.

[henk717]

Can you give us the text it shows at the top of the log after you launch with the use cpu option? It should be saying koboldcpp_default.dll

[no-connections]

If press run benchmark that is what I get. I used tiny llama cause it's fast to load but any other model is the same.

kobold_log.txt

If I select both no KV and set layers to 0 then I switch to Old CPU I get this:

kobold_kv_layer.txt

But if I only set KV or layer to 0 it crash. It has to be both.

[LostRuins]

Are you saying that after selecting vulkan and launching it crashes directly after printing

Power Throttling skipped in compatibility mode.
attach_threadpool: call
Starting model warm up, please wait a moment...

and then crash with no additional messages?

[henk717]

@LostRuins you might be accidentally defaulting to vulkan if its present for old cpu. We had this bug originally many releases ago for the regular, maybe it never got fixed for the old cpu build?
I can't reproduce this though, if I select that CPU backend it goes to the CPU backend even without changing anything else.

[no-connections]

I'm saying after selecting Use CPU (Old CPU) as per the image.

But selecting Vulcan crash as well:
kobold_vulcan.txt

edit I am using nocuda and oldpc builds interchangeably since it happen on both. Ill check the regular in a sec.
edit2 regular binary koboldcpp.exe does the same.