This is going to be a pretty subjective post. It will be mostly about special effects for fantasy and science fiction on television and in the cinema. What I like, what I don't like, what I wish they'd do and why.
I would say there are seven prose, non musical story telling media. They are cinema, television, theater, animation, comics, written prose and verbal story telling. The special effects that I am thinking of are mostly for the first two. While I will be focusing on special effects for fantasy and SF in TV and cinema I will make comparisons with other media and genres. (Super-hero and horror stories are fantasy sub-genres.) Some of my comments also apply to historical dramas.
Special effects are used in those visual story telling media which use live actors. They are used to give the illusion of something being there or happening. These are illusions of something that isn't available to the producers in real life (at least not yet) either because of cost (huge armies) , because they haven't been created yet (starships), they aren't around on Earth (aliens) or they don't exist (dragons).
Techniques available now include pure CGI, CGI traced over real objects, background mattes, models, scenery sets and prosthetics.
Special effects are of minor importance in theater. In part because science fiction and fantasy are uncommon genres in theater. Mostly because the resources available for special effects are limited in theater. This limitation is of course one of the reasons why SF and fantasy are uncommon genres in theater.
What the theater uses are props. These are objects and effects that the watcher can see are obviously not real but they guide the imagination of the watcher. They are a stylization that the audience and the producer tacitly agree on. The audience suppresses their disbelief in what they see and use the props as cues to their imaginations.
This is bit like the way one sees a novel or a spoken tale. These are not visual media but they can contain many cues for the visual imagination. Similarly one hears a comic. It is a silent medium but is rich in cues for the auditory imagination.
The distinction between an effect which suggests something and one which depicts it is not a hard and fast one. Look at space ships in a 1950s or earlier movie. There is a definite attempt to make the ship look real. But in many cases they didn't quite succeed. (Often they didn't come anywhere near succeeding.) Even more, look at the creatures. One needs to deliberately suspend one's reactions to the cues that say "This is not real.". The more realistic the special effects, the smaller this effort needs to be. With modern special effects the effort required is much smaller than with older shows. In an older show there were always some things that niggled. Nowadays the illusion is sometimes complete especially in the cinema or in a near future setting.
But not quite. There are things that cannot be truly depicted visually but they must attempt to do so. They use stylized depictions that while not strictly realistic tells the audience's imagination what is happening. The best example of this is the depiction of energy weapon beams in space. Of course they would actually be invisible. One can only see a beam passing through a material medium because of scattering, heating or ionization. This won't happen in a vacuum. But they have to be depicted somehow. We just allow the creators some artistic license and don't quibble about lack of realism.
Another example is psychic forces. These of course could not be seen but one has to use some form of visual depiction.
There are also things which could be depicted realistically but drama might be sacrificed in doing so. For example in a fleet action in space one would expect the ranges to be so great that one could hardly see enemy or friendly ships or more likely for all ships to be outside visual range of one another. But you have to see what is going on so ships are depicted as being unbelievably close to one another. I find it jarring but necessary (at least in some cases.).
Another way that realism is compromised is in the depiction of extra-terrestrials. They are almost all humans with funny heads. I can see why but I wish it was otherwise. First of all there is cost. It is cheaper to use an actor with prosthetics than to use models or CGI effects to depict a character. Second, it is easier for the audience to read the emotions of something with human body language, voice intonations etc. Something radically non-humanoid would be quite difficult to read. There seem to be more non humanoids in movies than on TV. Movies have bigger budgets. Also TV science fiction tends to rely less on spectacle and more on character than cinema. The smaller screen lends itself less to spectacle. The greater time available on TV allows for more emphasis on characters. With the greater emphasis on character comes a pressure to make the characters easy for the audience to read. Still it it jars for me. I'm too much of a biologist to find humanoid aliens believable. I know how variable life is and find that humans with funny heads undermine my suspension of disbelief. Trouble is, I can appreciate the economic realities. In many, perhaps most of these cases the choice is between aliens that are too human-like and having no aliens at all. Grumble!
There are two ways that a a depiction look wrong. It can depict the wrong thing or it can depict something unrealistically. Unrealistic depictions are usually either a lack of texture and depth or things moving jerkily or otherwise in the wrong manner. We are good at picking up something odd when looking at what is supposed to be a living being – especially if it is supposed to be human or human-like.
Backgrounds have improved enormously. Near future Earth settings are now usually very good both on television and in the cinema. So are the insides of spaceships and space stations and urban backgrounds. Settings for heroic fantasy are generally similar to to those for historical drama. Non urban backgrounds on other worlds are less satisfactory especially on television. Usually they just use some rural outdoors spot. These do not give any impression of being on another planet. The sky and scenery in the background can sometimes look a bit static, textureless and flat. The vegetation seldom is anything except the plants we know. Of course it would be difficult to create convincing alien appearing vegetation. Still more use could be made of CGI created vegetation, at least for movies. CGI can fail to convey detail and texture. However for something that is alien our brain and eyes will not be telling us that something is wrong as easily as they will if the CGI effects are used to depict something that is more familiar. This gives the creators a bit of leeway when depicting something unfamiliar.
Ships and other vehicles are now well depicted. In exploration centered space adventures such as 2001 A Space Oddessy they generally have been. I think the standard for appearance in a space combat adventure was set by the first Star Wars movie. That was when vehicles started having a lot of detail and texture and looking used. Before that they tended to look pristine with simple shapes. However if anything Star Wars set back the depiction of ship movement. The fighters moved like aircraft not space vehicles. Babylon 5 set the standard for ship movement. No banking, the direction a ship pointed had nothing to do with direction of movement and things had inertia. The problems with Star Wars et al. came from trying to import images and ambiance from other genres into a future where they didn't belong. They wanted World War 2 dogfights in space.
When depicting a battle there can be a conflict between conveying excitement and conveying suspense. If you want to convey excitement you use the viewpoint of someone in the middle of a melee. You rely on a quick succession of challenges. This can make it difficult to show the whole picture in a large battle. The emphasis is on the prowess of the hero.
Suspense requires the audience having time to realize what is happening. The audience may need to see the whole picture. In a large battle the viewpoint to use if one wants to build suspense is that of the commander or someone else outside the me lee. This can be difficult especially with a three-dimensional space battle. Perhaps this could be done by showing some three dimensional commanders display such as has been described in numerous SF stories. The only time I remember this being tried on TV was on Babylon 5. It was only a qualified success.
Software now allows the easy creation of crowds and armies and hordes of spaceships or aircraft without the traditional thousands of extras or models. Sometimes this lets the director show a horde which would be too expensive using extras. Sometimes it can lead to clutter and overkill. A huge horde rushing forward can be less scary than the sight of a smaller disciplined formation moving implacably towards you. I'm thinking here of the sight of the Roman legions in Spartacus. Another thing I don't like is showing armies in closely packed formations attacking in the face of automatic weapons or energy weapons. That is armies saying "Kill me!". Plenty of this nonsense in Star Wars movies, especially the prequels. I like to feel that the heroes deserved to win rather than had their victories decreed by the scriptwriters. Finally I have never seen a believable cavalry charge on screen. You do not gallop towards an enemy for a long period letting your formation get ragged. You approach at a walk, then a trot, then a canter and only gallop the last fifty yards or so. You hold the formation together for maximum impact. A realistic depiction of a cavalry charge would be far more menacing than what is shown on the screen.
End of moans.