Never taught elementary, but if he's slow on the uptake it wouldn't hurt to have him tested for learning disabilities, etc. Early intervention is the key to being undifferentiated from peers at older ages. On the other hand, its useful to recognize that at earlier ages girls tend to pick up complex concepts a little quicker than boys, so this could absolutely be within the realm of 'normal' for both of them.
Abstract concepts in general are difficult for young children to comprehend, not necessarily due to complexity inherent in the subject matter, but rather that they have not yet developed schemas for understanding the world in terms that we often take for granted. Jean Piaget, who practically pioneered the world of Developmental Psychology, was able to prove that young children are able to demonstrate abstract reasoning and logic when methods were demonstrated in terms they understand.
Related to this is the concept of learning styles. They can loosely be broken into three realms: visuo-spatial, auditory and tactile/kinesthetic. Without going into too much detail, each style represents the methods with which the individual learns best. Visual learners tend to need to see something demonstrated to grasp it. Auditory learners need to hear something explained to grasp it. Kinesthetic learners need to feel or 'experience' something to grasp it. Recognition of this is why schools organize curricula to encompass all three in concert. For example, teaching subtraction. First explain (auditory) the concept: you have three apples, when you take away one, how many remain? Then show it to them via diagrams or real fruit (visual), then finally let them physically have three apples and remove one to count the remaining group.
At any rate, testing will eliminate the possibility of a learning problem, or will alert you to it early, so that is probably your best bet early on. Otherwise, just stick to it. Many parents who homeschool run into this kind of problem with their own kids. Some kids just need more time to adjust to this type of supported learning.
Hope this helps...