5th October 2017 - "Cutting electrocautery versus scalpel for surgical incisions: a systematic review and meta-analysis."
Angus Hotchkies 10/05 07:36PM
Welcome to SCALPEL's third online journal club of the year! Thanks for coming along, hopefully it will be a useful discussion! We are joined by Daniel Lewis, a neurosurgical registrar who has kindly agreed to facilitate the discussion. I’m Angus Hotchkies, a 4th year based at Salford. Firstly if everyone wants to introduce themselves and include an email (so we can send certificates out later) that would be great! Here is a link to the paper for quick reference if anyone needs: https://www.sciencedirect.com/science/article/pii/S0022Welcome to SCALPEL's third online journal club of the year!
Sarah Michael 10/05 07:38PM
Hi i'm sarah a 3rd year at Wythenshawe
Bradley Storey 10/05 07:38PM
Hi I'm Bradley, final year at SRFT
Angus Hotchkies 10/05 07:38PM
Oops sorry! Ready to start when you are Dan!
Daniel Lewis 10/05 07:38PM
Okay great, how did you all find the paper?
Bradley Storey 10/05 07:39PM
Quite interesting read though lots of statistics/graphs/data that take a long time to get your head around (personally)
Sarah Michael 10/05 07:40PM
i thought it was a good paper from what i could see, they took a lot of variables into consideration when they got the results
Angus Hotchkies 10/05 07:40PM
I thought it was an interesting paper! Nothing statistically significant shown and agree with Bradley some fairly complicated stats!
Seemed to make some good arguments for electrocautery going forward
Daniel Lewis 10/05 07:41PM
I agree with you, its quite an extensive meta-analysis and hence lots of data and graphs. Firstly may be a silly question but are you all clear on the difference between a meta-analysis and a systematic review?
Sarah Michael 10/05 07:42PM
so systematic review collects the data? and meta analysis summarises and analyses it right?
Angus Hotchkies 10/05 07:42PM
the review collects the data and meta-analysis is doing the stats on the findings?
Bradley Storey 10/05 07:42PM
The use of all these statistics, essentially?
Daniel Lewis 10/05 07:44PM
Exactly, so reviews are essentially jut presenting the findings of different studies, they dont attempt a pooled statistical analysis. The advantage of meta analyses is you can pool data and hence increase the number of patients significantly, giving you more power to detect differences
What do you think of the way they identified articles for inclusion?
Sarah Michael 10/05 07:45PM
looks quite thorough, they said in two steps with three different people for each
Angus Hotchkies 10/05 07:46PM
yeah and they've excluded studies that only looked at one of the techniques
which seems fairly sensible
Bradley Storey 10/05 07:47PM
Yeah all their search criteria included lots
Daniel Lewis 10/05 07:47PM
I agree it was very thorough and must have been very time consuming. There was a significant number of exclusions as from 781 publications they ended up with only 41 studies
Crucially as well the majority (35) were RCTs
Have you come across the term quasirandom before?
Bradley Storey 10/05 07:48PM
Angus Hotchkies 10/05 07:48PM
Sarah Michael 10/05 07:48PM
Daniel Lewis 10/05 07:50PM
So its essentially an attempt at randomisation but using an allocation method that is not truly random
Crucially, it risks the investugator knowing which particpiants are in which treatment arm thereby increasing risk of bias
Angus Hotchkies 10/05 07:51PM
ok thanks - so it just increases the risk of bias which they mention in that section briefly
Sarah Michael 10/05 07:52PM
they mention risk of bias assessment?
Daniel Lewis 10/05 07:52PM
Yep, what was good about this paper is they did a formal Risk of bias analysis. And as expected the RCTs had lower risk of bias compared to the observational cohort studies
We'll come on to detecting various biases in a shortwhile. Let's talk about the individual results. What did you all think of the blood loss result?
Sarah Michael 10/05 07:56PM
most of the studies in the table said less blood loss i guess they concluded less blood loss overall because of that
Angus Hotchkies 10/05 07:57PM
So it's saying that electrocautery was associated with less blood loss - I'm not too sure how to interpret the random effects model though
Bradley Storey 10/05 07:58PM
Sorry yeah also not sure about random effects model
Daniel Lewis 10/05 07:59PM
So probably a good situation to talk about the different models. The issue with mat-analyses is you have multiple different data sets so as well as trying to decide if there a statistical difference between procedure A and procedure B, or drug A and drug B you also have to measure the heterogenity within the whole data set. High heterogenity means the studies disagree with each other on what the difference is
There are different ways to quantitatively assess heterogeneity and to model the data. If there is high heterogeneity then a random effects model is better
If heterogenity is low (ie the studies agree more with each other) than a fixed effects model is better
Sarah Michael 10/05 08:00PM
so its saying despite the high heterogeneity, the summary isn't that affected by those that disagree so the conclusion is the same?
Daniel Lewis 10/05 08:02PM
Yes essentially. To measure the degree of heterogeneity they have used a statistic called the I2 (there is another one - Cochrane's Q but its not very good). High I2 (94%) means high heterogeneity, and the differences between studies is significant (p<0.00001) so the differences between studies is not just due to chance
Is the difference between scalpel and diathermy still significant then. well yes beacuse the summary effect size (-1.16) is significantly significant , and when they omitted some studies the overall effect size was not altered
Angus Hotchkies 10/05 08:05PM
Ok that makes a bit more sense, thank you! I might have to read that over a few more times though!
Bradley Storey 10/05 08:06PM
Makes sense. Just to check, is the high heterogeneity between the methods/intended outcomes of studies or the actual results of A vs B?
Daniel Lewis 10/05 08:07PM
Good question, so the heterogenity is in the actual results- in this case the mean difference between diathermy and scalpel in terms of blood loss. The heterogenity in te results, is porbably caused by different methods etc
Another way to look at the dat is to look at the Forrest plot in figure A below
Forrest plots are a great way to see what the overall effect size is, AND assess heterogeneity in the data
Sarah Michael 10/05 08:09PM
oh yeah that makes the results more easy to understand thank you!
Daniel Lewis 10/05 08:10PM
Do you all understand what the lines and black diamonds mean on the Forrest plot are?
Angus Hotchkies 10/05 08:11PM
Bradley Storey 10/05 08:11PM
Sarah Michael 10/05 08:13PM
the diamond is the average isn't it?
Daniel Lewis 10/05 08:13PM
So the lines are the 95% CI for the individual studies, with the central dot being the mean standardized mean difference (SMD)
The central point of the black diamond is the mean cumulative effect size , or cumulative SMD. The two end point of the black diamond is the 95% CI of the SMD for the whole sample
You'll see as the number of studies gets bigger, the diamond becomes shorter as the CI becomes tighter
Sometimes they will also show the weighting of the individual studies by putting a box over the line
What about the funnel plot below?
Sarah Michael 10/05 08:16PM
ive never seen one before
Bradley Storey 10/05 08:16PM
More confusing than forrest plot
Angus Hotchkies 10/05 08:17PM
Daniel Lewis 10/05 08:18PM
So it took me a while to get my head around the funnel plot but once you understand them, they are extremely useful for assessing publication bias
The y axis is standard error of the individual studies, and the axis is different compared to normal graphs so that the low values are on top. Larger studies = lower standard error=more precise estimate of difference between diathermy and scalpel
So the more precise studies are near the apex of the inverted funnel whereas the studies with larger standard error are near the base
Does that make sense?
Angus Hotchkies 10/05 08:23PM
Bradley Storey 10/05 08:24PM
Angus Hotchkies 10/05 08:24PM
so the majority of the studies fell between 0.1 and 0.3
Daniel Lewis 10/05 08:25PM
Yep exactly, but the crucial thing to notice is that the funnel plot is symmetrical. This means that studies are either side of the overall mean difference, some saying diathermy is better or worse in terms of blood loss than the overall value
This is important - as it means there is very little publication bias here
Angus Hotchkies 10/05 08:26PM
ok I was just going to say I still wouldn't sure how to interpret it
If the results all were to one side, would that mean there was a definite publication bias or just a possibility?
Daniel Lewis 10/05 08:27PM
It they were all to one side it would strongly suggest there was a bias in the results - either reporter bias (researchers not reporting negative results) or publication bias (journal rejects negative results)
Angus Hotchkies 10/05 08:27PM
Daniel Lewis 10/05 08:28PM
If we were to do this with say studies on an expensive new drug - the funnel plot would likely be one sided, because drug companies dont like reporting negative studies
Bradley Storey 10/05 08:29PM
Angus Hotchkies 10/05 08:29PM
ahh very interesting
Daniel Lewis 10/05 08:29PM
Bradley Storey 10/05 08:30PM
Hehe, will make a note
Angus Hotchkies 10/05 08:30PM
Daniel Lewis 10/05 08:31PM
Back to this paper, is the heterogenity in blood loss surprising?
Angus Hotchkies 10/05 08:31PM
no because there's a range of surgeries being carried out?
Sarah Michael 10/05 08:31PM
i would have thought they would all show less blood loss for diathermy?
Bradley Storey 10/05 08:32PM
Not really as presumably different operations will have different blood loss depending on scalpel vs diathermy
and maybe even experience of the surgeon favouring one over other, not sure if that matters or already taken into account
Daniel Lewis 10/05 08:34PM
Exactly, so whilst they all show less blood loss (minus two studies) they're looking at different operations. What did you think about the incision time and hsopital stay data?
Angus Hotchkies 10/05 08:35PM
unsure how to interpret incision time. For hospital stay data surely there are far too many variables to interpret reliably?
Bradley Storey 10/05 08:36PM
Agreed and thats probably why they don't really comment on hospital stay?
Just a question for my understanding, why would incisional time in seconds matter?
Daniel Lewis 10/05 08:38PM
Its a good question, there are very few situation s where saving seconds on an incision would generally matter - hosing rupture AAA, rapidly expanding extradural maybe?
For daily practice though, the decreased blood loss and incision time needs to be balanced against the increased risk of seroma in this meta-analysis
Bradley Storey 10/05 08:41PM
Mm yeah I agree I think
Daniel Lewis 10/05 08:42PM
A few final points before we close, on the seroma dat the i2 value is quite low meaning that almost all the studies were homogenous in saying the risk of seroma was higher
Would this meta-analysis change anyones management?
Angus Hotchkies 10/05 08:45PM
Sorry as in the meta-analysis for the seroma or overall?
Bradley Storey 10/05 08:45PM
I doubt it given that I don't think the values measured seem to be the most clinically relevant of things, especially if complication risk (seroma) higher. Also most surgeons probably stick to what they prefer and have experience in? Finally though I'm not that aware of every journal, if it was more important findings - even though a well researched study - wouldn't it be published in a more recognised journal; I'm not familiar with the Journal of Surgical Research?
Angus Hotchkies 10/05 08:48PM
Overall, I think it's great for students to read as it makes us think about what we might use for incision should we become surgeons. I agree with Bradley in that surgeons will likely use their experience primarily but further studies may change managment in the future and this paper does seem extremely thorough!
Daniel Lewis 10/05 08:50PM
I agree with you both, I think their statistical methods are very thorough and a nice example of how do a meta-analysis. Will it change practice, probably not. Might have done away with the higher infection risk claim from diathermy but otherwise icnision time and blood loss (?clinically significant) are unlikely to change individual surgical practice
Bradley Storey 10/05 08:52PM
Yeah, great thank you
Angus Hotchkies 10/05 08:52PM
Yeah it seems like a good example of systematic review and meta-analysis to refer to! Thanks for breaking it down and making it easier to understand!
Sarah Michael 10/05 08:53PM
yeah thank you!
Daniel Lewis 10/05 08:54PM
No worries, glad it was useful
Bradley Storey 10/05 08:54PM
It was thanks again!
Angus Hotchkies 10/05 08:55PM
Thank you very much!
Daniel Lewis 10/05 08:55PM
See you all the next one, I best go before my other half takes a scalpel to me for being late!
Angus Hotchkies 10/05 08:56PM
Sounds like you better - cheers for your time
Angus Hotchkies 10/05 08:58PM
cheers guys - certificates sent to you both
Sarah Michael 10/05 08:59PM