Yes, there are even scriptures in the bible to sort of support the idea your child is in heaven.
Children will not be in heaven because they are innocent. Paul taught clearly that children are born under condemnation of Adam’s sin (Rom. 5:12) . . . . If children are saved (and I believe they shall be) it can only be because God credits their sin to Christ; and because they are too young to believe, the requirement of personal faith is waived. The clear teaching of the Bible is that the souls of those children who die before they reach the age of understanding and responsibility are taken to the New Jerusalem with Christ to live joyfully with Him until the day of resurrection when they shall receive their own immortal body fit for eternity. I believe the Bible shows us that children will be in heaven. —GRANT JEFFREY
Wiese, Bill. 23 Minutes In Hell: One Man’s Story About What He Saw, Heard, and Felt in That Place of Torment (p. 112). Charisma House. Kindle Edition.