In this part of the conversation, Jim is shocked to learn that the man isn't sympathetic and helpful. Kirk becomes angry when this guy, whom he formally respected, refuses to even try to think of a way that the thing can be done.
Jim decides that he doesn't owe him any loyalty when the big cheese trivializes his great love for Spock, and dismisses his desperation as emotionalism. If Starfleet doesn't understand his loyalty to Spock, then they don't know what loyalty is, and won't miss it if he withdraws his!
As we watch, Kirk makes the radical decision to defy orders. He doesn't yet know what he will do, or if anyone will help him, all he knows is: FUCK STARFLEET, he's going.
Kirk would not be thinking FUCK STARFLEET, if Starfleet hadn't in effect said FUCK YOU to him, first. He feels that he deserves more support after his many HONORABLE years of service, sacrifice, and toil. He certainly thinks that Spock deserves more.