Don’t be down on yourself!! Remember that if you are having a hard time and are stressed out (and I can only imagine dealing with this as a single parent must be extremely stressfull)that your baby can sense that. Whatever mood you are in is going to rub off on your baby.
Once you get ahold of yourself and take back home, you will see a difference (for the better). Just remember, if your baby is upset, they are going do better with someone who is calmer (usually not the mother, we tend to get upset right along with them, it’s only natural).
I wish you the best of luck, and hope things get better.