members not allowed at HOA meetings

MAY I, AS PRESIDENT OF OUR HOME OWNERS ASSN, RESTRICT OUR
MONTHLY MEETINGS TO THE BOARD MEMBERS ONLY? ALSO, MAY I
DENY MEMBERS ACCESS TO OUR MINUTES OF THE MEETING? OUR TOWNHOMES ARE OCCUPIED ABOUT 70 PERCENT WITH RETIRED PEOPLE OR THOSE OVER THE AGE OF 60 - WE RECENTLY HAVE HAD SOME PEOPLE MOVE IN WITH SMALL CHILDREN AND WE HAVE PASSED A RULE THAT CHILDREN ARE ALLOWED IN THE POOL FROM 10 AM UNTIL 7 PM - WE HAVE HAD A LOT OF COMPLAINTS ABOUT THIS - DO WE HAVE THE RIGHT, AS BOARD MEMBERS, TO MAKE THIS RULE ABOUT POOL HOURS?

You are asking more than one question here. Your states rules and governing documents are the first place to look.
In our state we are not allowed to restrict members from attending our Board meetings and we prefer to have our members attend. However, we do not allow our members to speak during our meetings.
We also have a pool and are a mixed age range community with small children and retired adults. We provide an adult swim time from 9am-10am every morning and this has been well-established for a number of years and utilized by many and griped about by others.

most rules are about the same, you have to let owners in good standing attend the meeting, unless the meeting itself is "closed" in order to discuss litigation, personnel issues, or some other thing that requires confidentiality by law.

However, as noted, attendance is open to all... speaking and having any power is not. Just because someone attends does not mean they get to vote, speak out, argue, etc.. Your meetings should be clear and open, there is nothing to hide. However, you don't have to allow time for the floor to speak ad infinitum on issues, etc.. they can sit and watch.