searching doesn't really work, I looked for Terraria and nothing showed up (although it could still be indexing)
Yeah, it won't do that for some reason. It doesn't index program files by default, but even enabling that won't help you. I actually specifically told it to index the folder where Terraria.exe is held and it wasn't fixed.
The issue is that the Steam game shortcuts are listed as Internet shortcuts, so it doesn't seem to list them correctly.
Here is a workaround:
Go to the following directory: C:\Users\NAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Steam
Create a .vbs file (Visual Basic Script)
Enter the following code
Set WshShell = CreateObject("WScript.Shell" )
WshShell.Run chr(34) & "steam://rungameid/105600" & Chr(34), 0
Set WshShell = Nothing
You can replace "steam://rungameid/105600" with any of the steam shortcuts.
Create a shortcut to the .vbs file in the same directory
This shortcut will now show up in search results and will open up your Steam game
It's all kind of tedious, but it's what I got.