Viruses can infect only certain species of hosts and only certain cells within that host. Cells that a virus may use to replicate are called permissive. For most viruses, the molecular basis for this specificity is that a particular surface molecule known as the viral receptor must be found on the host cell surface for the virus to attach. The viral receptors among different species can be highly similar but there are always some difference in their DNA sequence so it is usually difficult for one virus to affect different species with no mutation.  

Example, bat virus need to be attached to the receptors on the bat cells (say ACE2 here), bat ACE2 is different from human ACE2 (DNA sequence difference) but maybe ocassionally bat virus can weakly bind to human cells.  If in this case the virus gets into a cell, most likely the virus will be killed by human immune response.  But if it survives and a mutation occurs which makes the new mutated virus bind to human cells more efficiently, then the virus may be reproduced and become dangerous.  I use bat and human as example which is not quite right because the sequence is very different, that is why people think there is a mediator.