Traditional bridge surface damage detection primarily relies on visual inspections conducted by humans, allowing only qualitative judgments of the type of damage or rough estimates of its scale. This method cannot accurately calculate the size of the damage or quantitatively assess its severity, making it challenging to formulate appropriate maintenance strategies. Based on this issue, a method for qualitative detection and precise quantitative assessment of bridge surface damage is proposed, utilizing deep learning and binocular vision. Firstly, stereo images of the damage are captured using binocular cameras, and the distance information is calculated based on stereo vision. Then, a qualitative diagnosis of the damage type is performed using an object detection model, which outputs the detection box for the damage location. Subsequently, an optimization strategy based on a buffer zone is proposed to accurately extract the outline edges of the damage. Finally, the size characteristics of the damage are quantitatively calculated based on stereo distance information and the damage outline, enabling the assessment of its extent. The method's feasibility and evaluation accuracy are validated using a river bridge in a specific area of Shanghai Pudong as an example. The results indicate that the proposed method can rapidly and accurately determine the extent of surface damage on bridges, effectively guiding bridge maintenance work.