Shcherbakova scored 255.95 overall to seal the title, while Valieva, who was allowed to compete despite failing a doping test before the Games, fell several times in Bejing to slip out of the medal positions.
Another Russian, Alexandra Trusova, took silver, and Japan’s Kaori Sakamoto won bronze.
The International Olympic Committee (IOC) had said if Valieva placed in the top three, no medals would be awarded until “due process” had been followed to decide if she had infringed the World Anti-Doping Code.
But given the results, the medal ceremony went ahead.
Shcherbakova, 17, said: “I still haven’t realised that my Olympic Games have ended. I just know that I skated clean. I am so happy that I still haven’t realised the result.”
It was a major upset for Valieva, who looked completely broken at the end of her programme.
She had been the clear favourite going into the Games, but a week in it emerged she had tested positive in December for trimetazidine, banned for athletes because it can increase blood flow efficiency and help endurance.
The Court of Arbitration for Sport (CAS) on Monday ruled that she could carry on competing in the Chinese capital.
Dressed in black and red, Valieva looked focused as she powered onto the ice, cheered on by the crowd, as well as a large contingent of her teammates in the athletes’ stand.
But she failed to land multiple jumps she had completed with ease when she performed the same “Bolero” programme in the team event earlier at the Games.
She was awarded 141.93 for it – almost forty points off what she took last week.
Supported by her coaches, she walked seemingly stunned to the kiss and cry area, where athletes wait to receive their scores, crying.
She did not speak to media as she left the arena.
Skating just before Valieva, Shcherbakova’s lithe, elegant movement and two successful quad jumps had pushed her into the lead.
Dressed in a shimmering burgundy dress and skating to a dramatic classical medley, Shcherbakova pumped her fists in celebration as the music died.
Second-placed Trusova, who included five quad jumps in her routine – though not all were landed cleanly – had a total score of 251.73.
Sakamoto, who had come into the Thursday’s free skate ahead of Trusova, ended with 233.13.
Before news of the doping test broke, Valieva helped the Russian team win the Olympic figure skating team title, but the medals for that event will not be awarded during these Games as Valieva’s test must be investigated.