No, you are getting the game all wrong.
There are two footballs. And there are 4 teams.
The 4 teams are divided into two groups, and there is a sort of alliance between the teams withing the two groups.
The purpose of the game is to get the football to the other end of the field, and throw it down on the ground hard. This is called a touchdown.
But the rules are, that you are only allowed to run with the ball, or catch the ball, if there are at least 2 people from your team (your actual team, not your group), within 20 feet of you. That is why you see the little groups fighting and running this way and that way–they are trying to arrange to have 2 of them within the 20 feet.
Here is where the alliance comes in–both teams within the group get points when a member of that group does a touchdown. So they try to help each other have the requisite 2 members. However, the points are not awarded evenly. Each touchdown wins you 3 points and the team which actually did it gets 2 while the other team gets only 1. But, then the team which got 1 gets to try to throw the ball through the upright poles in order to get 1 point. This is called “punting”, because one team in the group “punts” it to the other team in the group allowing them to get 2 points and hoping they will make it up by throwing it through the uprights. (You are also allowed to kick it through the uprights, and that is called a “kick-off”.)
Now, there are two balls, as I mentioned, and both balls are being played with by all the teams at the same time. On TV, they only show one at a time obviously, but that is how sometimes the score can change by more than 4 very quickly.
There are some other things; I’ll mention just a few.
“Out of bounds”. That is when one group is winning by a lot, and they begin to turn against each other, since they are really the threat to each other. This is called playing “out of bounds” because you are supposed to play as a group.
“Interception”. This is when one team within the group pretends they are helping the other team within their group, but then one of them sneaks in and grabs the ball.
“Instant replay”. This is when you do the same play again right away, figuring the other group won’t be expecting it. They usually show it on TV in slow motion, because they want you to see how similar it it.
There are also bludgers and quaffles, but that is very involved, and they aren’t always used in every game.