Silvergate shares have fallen this week on broader crypto market worries, including the fact that FTX is a customer of theirs.