梁職貢圖 is an interesting ancient record. The original 梁職貢圖 was made by 蕭繹 during the Liang Dynasty in China. The original record was then copied during the Northern Song. This copy was copied again by a Qing artist 張庚. This version is called 諸番貢職圖. 愛日吟廬書畵續錄 (1914) contains it. This edition discovered by 趙燦鵬 includes 題記 for Silla and Goguryeo, too. 梁職貢圖 舊藏本 in Nanjing Museum is an incomplete edition that includes only Baekje 題記 for Korean kingdoms. Another edition called 唐閻立本王會圖 kept in a museum in Taiwan (故宮博物院) contains drawings for all three envoys but it omits 題記.
This is 梁職貢圖 舊藏本 that describes various foreign envoys.
This is the part for Baekje.
Baekje, Goguryeo and Silla envoys depicted in 唐閻立本王會圖.
This is the newly discovered 新羅題記 from 愛日吟廬書畵續錄.
There are small errors. 姓募名泰 should be 姓募名秦 and 健年 should be 健牟羅. It also doesn't make sense at the basic level that Silla wasn't using letters in the 6th C. At that time, the Silla envoy was depending on the Baekje envoy for translation and Silla was a competing country for Baekje. It's highly likely that the Baekje envoy didn't describe Silla favorably. The phrase the OP is highlighting is this: 或屬韓或屬倭國. "(Silla) belonged to either 韓 or 倭." As with many ambiguous and contradictory lines in ancient records, there is a lot of room for how to interpret it. The fact is that Silla and Baekje fought for dominance and Wa (Yamato) was always allied with Baekje, providing military aid. The borderlines were constantly changing and Wa was involved in many wars. In the beginning, Silla was the weakest of the three kingdoms in the peninsula but Silla managed to expand its territory. Silla's 眞德女王 allied with Tang and eventually conquered the other kingdoms in the peninsula.