"If one does not know to which port one is sailing, no wind is favorable." - Seneca
We can't hope to craft a toolbox meant to help us write good stories if we can't define what a good story is. And though it's an ambitious and endlessly debatable question, I'm going to propose a rather simple answer. We can define whether a story is "good" from a utilitarian viewpoint: does the story accomplish its job? I propose that a story has two primary jobs.
"Story as such can only have one merit: that of making the audience want to know what happens next. And conversely it can only have one fault: that of making the audience not want to know what happens next.” - E. M. Forster. “Forget narrative, backstory, characterization, exposition, all of that. Just make the audience want to know what happens next.” - David Mamet
The first job of any good story is to make the audience want to know what happens next. The story should be compelling. It should be interesting. It should be unquestionably engaging. And if this piece doesn't exist, don't count on an ending that will blow the audience's mind because they'll never get there.
“I think we’ve all had that experience of going to see a movie and walking out of an ending and thinking, ‘My God, that ending was so great!’ and you have this sense of euphoria, release, this sense of clarity, and you feel like you’re looking at the world with new eyes.” – Michael Arndt
The second job of any good story is to make the audience see either themselves or the world with brand new eyes. This is no easy feat. It's what causes us to bring a story home to the dinner table. It keeps the story bouncing around in our head the entire week, long after we've left the theatre. It's the most difficult goal to accomplish but it's well worth it if you can pull it off.
In short, a good story has two jobs:
Keep the audience interested
Change the audience's perception of either themselves or the world
Give the story meaning
The audience approaches each story, then, with two demands: "Keep me interested. Change me." The journey and the destination. Compelling and life-changing.
The tools in our toolbox must be in service of these two story goals. We can call those tools that attempt to keep the audience interested "dramatic". We can call those tools that attempt to change the audience's perception "thematic".
Our dramatic tools will help us answer questions like: "How do I keep the audience in suspense?" "How do I make the audience care about a character?" "How do I increase tension?" "Why is more conflict usually better?" "What's the difference between implicit tension and explicit tension?" "How do I write interesting dialogue?" "How do I create subtext?" "How do I keep the audience guessing?" "How do I setup expectations?", and more.
Our thematic tools will help us answer questions like: "How do I create a character arc?" "What creates meaning in a story?" "How do I make a moral argument?" "What are philosophical stakes?" "How do I structure a story?" "How do I build a story world?" "How I use symbolism?" "How do I create characters?" "How do I decide what my characters should do?", and more.
This definition of what makes a good story is subjective in the sense that the observer decides whether they were interested and whether their perception has been changed by the story. But we're dealing in the realm of art. As has been said, art exists only in the eye of the beholder. It is a subjective endeavor by its very nature and so our definition of story must necessarily be centered around the subject. There is, fortunately, general consensus in the classification of certain art as "bad" and certain art as "good".
Given that story is subjective, our goal isn't to ensure that a story performs both of its jobs with 100% efficiency. If we can keep most of the audience interested, and if we can change the perception of most of the audience, we will have won. That's our goal, and these are our metrics. Fortunately measuring whether an audience member is interested is fairly straightforward--is the audience member engaged? Are they mentally stimulated? Are they participating in the story? Each audience member can generally tell us whether they were bored, confused, or interested. We'll take that as a good metric of our success in the first job of keeping the audience interested.
The second job of a story--changing the audience's perception of themselves or the world--is a bit more intangible. It will be harder to measure with certainty but let's clarify our goal. We want the audience to walk out of the movie thinking, "Wow. I feel different." That's it. We want to have a marked effect on the audience in such a way that they encourage their friends and family to see the movie (hopefully not because it was terrible! But hey, at least we'll still sell tickets).
And so this is how we'll measure whether a story is good. It's an endlessly debatable question, but we've got a solid start.