This has more to do with the goal of the techniques. PCA in bulk RNAseq is intended for QC to see if there are outliers. tSNE is used in scRNA-seq is used to find cell-types or other groups. You could use tSNE in place of PCA in bulk RNAseq, but since there are parameters to tweak and it's more computationally expensive there's no great benefit to do so.
My impression is that one of the main reasons is the simple fact that tSNE hasn't been around as long as PCA (Reference) Plus, PCA tends to work well on bulk RNA-seq data and mathematically, its application for detecting outliers as Devon pointed out, makes sense since it's calculating the vectors along which the variation is maximal -- in bulk RNA-seq, you typically want the variation to stem from experimental factors rather than from individual samples, so PCA is a good check for that. In scRNA-seq you typically don't have many different samples, instead you have thousands of different cells, usually stemming from only one or a handful of different samples.
I think this is largely because how the clusters are distributed in sample space. For instance, in cancer research we usually have overlapping Gaussian clusters. Often quite a simple structure. Both PCA and tSNE work fine to show these structures in my experience. Sometimes, but rarely, the structures in some datasets may be more complex, towards single cell RNA-seq complexity, and tSNE works better in these situations.
In single cell RNA-seq oftentimes we have far more complex structures usually consisting of many globular clusters (cell types) of different sizes and variance arranged in complex patterns in sample space. tSNE can capture complex non-linear structures well, PCA can't.
Edit: you may want to look into the UMAP algorithm.
I think the main advantage to tSNE is that it will space out the data points, but a disadvantage can be you may sometimes want to be careful about over-interpreting some swirly shapes within the tSNE plot (and PCA is often OK with the smaller number of samples for RNA-Seq project). However, Devon is also correct that tSNE is often used for separting cell types (and I would expect that to be increasingly true as larger numbers of cells/samples are considered).
It may also be helpful to review Difference between tSNE and PCA analysis, which I used to confirm that PCA is used as an upstream step for tSNE.
I think tSNE is capturing the local relationships between points, like treating your data as a network where your cells are nodes. PCA is calculating the "true" distances between points after we consider variation in your dataset. Mahalanobis distance can do similar things.
Applying PCA before tSNE is in fact projecting your data into a low-dimensional subspace, where the distances between points are more real and therefore you could obtain more real local relationships between points.