Hi and welcome to Discuss Cooking
Do you cool and refrigerate the soup the same night you make it? Leaving it out overnight at room temperature runs the risk of bacteria growing in it which could cause it to fizz and become sour.
You can cool it more quickly by putting it into smaller containers and then putting those in a sink filled with ice water. It needs to be below 70 degrees F before you refrigerate it.
Also, you don't need to boil the soup. Bring it to a boil, reduce the heat to low and let it simmer for an hour or till the chicken is ready to shred. Shred it and return it to the soup. Now you can turn it off and let it cool, using the method I mentioned above.
Hope this helps.