When you engage manual sort order it still puts information into order on the Kanban report based on the sort order of the original field being used (For example if it was in order by Date Created) but then once manual sort is enabled any records that are specifically pulled out of order, in your example record six being pulled to be the 1st in priority will now going forward be in place as the first record in that order while the other records will continue to use the previous sort until they are pulled manually into a different sort. The previous sort order exists as a stand in until you chose to take action to change the records sort order but it wouldn't change the Date Created on that record just the records place on your Kanban report.