I often hear this. It's a common thought and internal struggle among those betrayed by their spouses. Their emotions and heart are devastated. Every negative thought possible can run through the head as the hurt spouse realizes he or she is married to a cheater. In effect, the suffering spouse may carry the blame or feels it's her/his fault her/his spouse is unfaithful.
If you're the betrayed spouse, here's the truth: you hold zero responsibility for the infidelity itself. No matter how serious your marital problems were prior to the cheating, the adultery trigger was pulled by your husband or wife, not you. It's a character pathology/deficit issue. Accept responsibility for your part of the breakdown within your marriage. But not for the affair itself. That choice rests 100% on your spouse. Any problems in marriage can be addressed in healthy ways, such as therapy/counseling, seeking support and help, etc. Yet your cheating spouse chooses otherwise, which makes the problem a hundred-fold greater than what you started out with.
Right now, your primary responsibility post-affair is to take care of your self. First and foremost, you need to save and rebuild your self before you can have the strength and capacity to heal your marriage if it's still possible ... or, move on as a better person. If you don't, it's going to be very harmful and disabling to you mentally and emotionally. It also affects your physical health. You need to work on psychological/spiritual issues such as self-esteem, self-doubts, lack of energy, love you have for your children, meaning of life.
During this sensitive time, one person matters the most - YOU.