Actually.. thinking back on it now; when he first used his bankai he progressively got slower as his bones were getting "crushed by the speed" now we all know that byakuya is incredibly fast (though in some fillers it just wasn't that much)
but trying to get back on point - maybe his body is rejecting the abilities of the bankai is it could destroy it?
With plot holes... 1) His hollowy take over had no problem with it
2) Hoes does a body reject a bankai?
Yeah it's a completely flawed theory but.. there's just no helping a completely useless bankai.. though it's not that useless.. it makes a really big sword.. smaller

tada!! it's imba

lol but meh I refer to my previous post - that's why it's pointless now, however, it's quite fun to think about isn't it? :P