The first answer is slow
The first request loads the weights from disk into memory, so it can take seconds to a minute depending on the drive. Later requests reuse them. If every answer is slow, the model is being swapped in and out — that means the machine does not have enough RAM for the quant you chose.