I have been using a Standalone AI at home. By Standalone I mean it needs no internet access.
This is private and secure. It also means if you run it on a laptop, in a blackout you still have access to AI's smarts as long as the laptop has power.
I use Ollama on Windows. This program loads a model with specialised knowledge, like medicine, or general knowledge. I find most models can answer most questions, though specialised models will have greater knowledge in their area of expertise.
First steps:
Download Ollama here
Now you need to download models to load. They are searchable on the same Ollama download page.
Hint: If you have an older PC and don't have an AI chip in your computer, try the lightweight models as the bigger ones can be very slow to answer questions. By slow I mean many seconds to minutes to reply. If unsure you will find out soon enough when you ask questions.
The following commands are run in a powershell window.
To download a model and run it:
$ ollama run <some model>
eg. ollama run llama3.2
To download a model but not run it:
$ ollama pull <some model>
To delete a model:
$ ollama rm <some model>
To find out what models you have installed:
$ ollama list
To update the models you have downloaded previously:
ollama list | Select-Object -Skip 1 | ForEach-Object {
$model = ($_ -split '\s+')[0]
if ($model -and $model -ne "NAME") {
Write-Host "Updating model: $model"
ollama pull $model
}
}
