Scaling does NOT turn your 4K screen into a smoother 1080p screen. Fonts are the perfect example modern fonts aren't pixels they are resolution independent descriptions of shapes. One could draw an S as small as it appears as one you see in this comment or as big as your screen. As its drawn bigger and bigger the curve is drawn with more and more pixels and the letter has the same form and shape at any size.

Whereas monitors range from 68-360 dpi you probably CAN read a 12 pt font at around 6mm on a monitor 12-18 inches away but probably CAN'T read it at 1mm because your eyes just can't resolve or can't comfortably resolve shapes below a certain size even with infinite pixels. It is possible you can read slightly smaller text which you can achieve by slightly tweaking font size in addition to scaling.

EG you scale your 4K monitor at 2x but decrease font size from 12 -> 10 pt or whatever combination produces a pleasant usable experience for you.

Similarly graphics aren't degraded. Your 4K video isn't scaled down to 1080p because you are scaled at 2x. Your environment gives your video player a rectangle to fill with pixels. The controls drawn by the app are scaled the contents are the contents of the video file as decoded. The same is true of graphics but because graphics aren't resolution independent it depends on high enough resolution assets being available. Your browser given a 400x400 space to draw an image linstead of a 200x200 will draw a nicer image if one is available.

TLDR: Scaling makes high dpi displays look nicer not shittier. Integer scaling is nicer and simpler but fractional scaling is needed for some less common resolution and size combos. Setting scaling and then tweak font size to taste.