Quote from Scalpel on 7th October 2017, 11:53 am

Angus Hotchkies 10/05 07:36PM

Welcome to SCALPEL’s third online journal club of the year! Thanks for coming along, hopefully it will be a useful discussion! We are joined by Daniel Lewis, a neurosurgical registrar who has kindly agreed to facilitate the discussion. I’m Angus Hotchkies, a 4th year based at Salford. Firstly if everyone wants to introduce themselves and include an email (so we can send certificates out later) that would be great! Here is a link to the paper for quick reference if anyone needs: https://www.sciencedirect.com/science/article/pii/S0022Welcome to SCALPEL’s third online journal club of the year!

 

Sarah Michael 10/05 07:38PM

Hi i’m sarah a 3rd year at Wythenshawe

 

Bradley Storey 10/05 07:38PM

Hi I’m Bradley, final year at SRFT

 

Angus Hotchkies 10/05 07:38PM

Oops sorry! Ready to start when you are Dan!

 

Daniel Lewis 10/05 07:38PM

Okay great, how did you all find the paper?

 

Bradley Storey 10/05 07:39PM

Quite interesting read though lots of statistics/graphs/data that take a long time to get your head around (personally)

 

Sarah Michael 10/05 07:40PM

i thought it was a good paper from what i could see, they took a lot of variables into consideration when they got the results

 

Angus Hotchkies 10/05 07:40PM

I thought it was an interesting paper! Nothing statistically significant shown and agree with Bradley some fairly complicated stats!

Seemed to make some good arguments for electrocautery going forward

 

Daniel Lewis 10/05 07:41PM

I agree with you, its quite an extensive meta-analysis and hence lots of data and graphs. Firstly may be a silly question but are you all clear on the difference between a meta-analysis and a systematic review?

 

Sarah Michael 10/05 07:42PM

so systematic review collects the data? and meta analysis summarises and analyses it right?

 

Angus Hotchkies 10/05 07:42PM

the review collects the data and meta-analysis is doing the stats on the findings?

 

Bradley Storey 10/05 07:42PM

The use of all these statistics, essentially?

 

Daniel Lewis 10/05 07:44PM

Exactly, so reviews are essentially jut presenting the findings of different studies, they dont attempt a pooled statistical analysis. The advantage of meta analyses is you can pool data and hence increase the number of patients significantly, giving you more power to detect differences

What do you think of the way they identified articles for inclusion?

 

Sarah Michael 10/05 07:45PM

looks quite thorough, they said in two steps with three different people for each

 

Angus Hotchkies 10/05 07:46PM

yeah and they’ve excluded studies that only looked at one of the techniques

which seems fairly sensible

 

Bradley Storey 10/05 07:47PM

Yeah all their search criteria included lots

 

Daniel Lewis 10/05 07:47PM

I agree it was very thorough and must have been very time consuming. There was a significant number of exclusions as from 781 publications they ended up with only 41 studies

Crucially as well the majority (35) were RCTs

Have you come across the term quasirandom before?

 

Bradley Storey 10/05 07:48PM

Not previously

 

Angus Hotchkies 10/05 07:48PM

No

 

Sarah Michael 10/05 07:48PM

nope

 

Daniel Lewis 10/05 07:50PM

So its essentially an attempt at randomisation but using an allocation method that is not truly random

https://www.eupati.eu/glossary/quasi-randomised-trial/

Crucially, it risks the investugator knowing which particpiants are in which treatment arm thereby increasing risk of bias

 

Angus Hotchkies 10/05 07:51PM

ok thanks – so it just increases the risk of bias which they mention in that section briefly

 

Sarah Michael 10/05 07:52PM

they mention risk of bias assessment?

 

Daniel Lewis 10/05 07:52PM

Yep, what was good about this paper is they did a formal Risk of bias analysis. And as expected the RCTs had lower risk of bias compared to the observational cohort studies

We’ll come on to detecting various biases in a shortwhile. Let’s talk about the individual results. What did you all think of the blood loss result?

 

Sarah Michael 10/05 07:56PM

most of the studies in the table said less blood loss i guess they concluded less blood loss overall because of that

 

Angus Hotchkies 10/05 07:57PM

So it’s saying that electrocautery was associated with less blood loss – I’m not too sure how to interpret the random effects model though

 

Bradley Storey 10/05 07:58PM

Sorry yeah also not sure about random effects model

 

Daniel Lewis 10/05 07:59PM

So probably a good situation to talk about the different models. The issue with mat-analyses is you have multiple different data sets so as well as trying to decide if there a statistical difference between procedure A and procedure B, or drug A and drug B you also have to measure the heterogenity within the whole data set. High heterogenity means the studies disagree with each other on what the difference is

There are different ways to quantitatively assess heterogeneity and to model the data. If there is high heterogeneity then a random effects model is better

If heterogenity is low (ie the studies agree more with each other) than a fixed effects model is better

 

Sarah Michael 10/05 08:00PM

so its saying despite the high heterogeneity, the summary isn’t that affected by those that disagree so the conclusion is the same?

 

Daniel Lewis 10/05 08:02PM

Yes essentially. To measure the degree of heterogeneity they have used a statistic called the I2 (there is another one – Cochrane’s Q but its not very good). High I2 (94%) means high heterogeneity, and the differences between studies is significant (p<0.00001) so the differences between studies is not just due to chance

Is the difference between scalpel and diathermy still significant then. well yes beacuse the summary effect size (-1.16) is significantly significant , and when they omitted some studies the overall effect size was not altered

 

Angus Hotchkies 10/05 08:05PM

Ok that makes a bit more sense, thank you! I might have to read that over a few more times though!

 

Bradley Storey 10/05 08:06PM

Makes sense. Just to check, is the high heterogeneity between the methods/intended outcomes of studies or the actual results of A vs B?

 

Daniel Lewis 10/05 08:07PM

Good question, so the heterogenity is in the actual results- in this case the mean difference between diathermy and scalpel in terms of blood loss. The heterogenity in te results, is porbably caused by different methods etc

*probably

Another way to look at the dat is to look at the Forrest plot in figure A below

Forrest plots are a great way to see what the overall effect size is, AND assess heterogeneity in the data

 

Sarah Michael 10/05 08:09PM

oh yeah that makes the results more easy to understand thank you!

 

Daniel Lewis 10/05 08:10PM

Do you all understand what the lines and black diamonds mean on the Forrest plot are?

 

Angus Hotchkies 10/05 08:11PM

black diamonds?

 

Bradley Storey 10/05 08:11PM

Yeah

 

Sarah Michael 10/05 08:13PM

the diamond is the average isn’t it?

 

Daniel Lewis 10/05 08:13PM

So the lines are the 95% CI for the individual studies, with the central dot being the mean standardized mean difference (SMD)

The central point of the black diamond is the mean cumulative effect size , or cumulative SMD. The two end point of the black diamond is the 95% CI of the SMD for the whole sample

You’ll see as the number of studies gets bigger, the diamond becomes shorter as the CI becomes tighter

Sometimes they will also show the weighting of the individual studies by putting a box over the line

What about the funnel plot below?

 

Sarah Michael 10/05 08:16PM

ive never seen one before

 

Bradley Storey 10/05 08:16PM

More confusing than forrest plot

 

Angus Hotchkies 10/05 08:17PM

same

 

Daniel Lewis 10/05 08:18PM

So it took me a while to get my head around the funnel plot but once you understand them, they are extremely useful for assessing publication bias

The y axis is standard error of the individual studies, and the axis is different compared to normal graphs so that the low values are on top. Larger studies = lower standard error=more precise estimate of difference between diathermy and scalpel

So the more precise studies are near the apex of the inverted funnel whereas the studies with larger standard error are near the base

Does that make sense?

 

Angus Hotchkies 10/05 08:23PM

yeah

 

Bradley Storey 10/05 08:24PM

Sort of

 

Angus Hotchkies 10/05 08:24PM

so the majority of the studies fell between 0.1 and 0.3

standard error

 

Daniel Lewis 10/05 08:25PM

Yep exactly, but the crucial thing to notice is that the funnel plot is symmetrical. This means that studies are either side of the overall mean difference, some saying diathermy is better or worse in terms of blood loss than the overall value

This is important – as it means there is very little publication bias here

 

Angus Hotchkies 10/05 08:26PM

ok I was just going to say I still wouldn’t sure how to interpret it

If the results all were to one side, would that mean there was a definite publication bias or just a possibility?

 

Daniel Lewis 10/05 08:27PM

It they were all to one side it would strongly suggest there was a bias in the results – either reporter bias (researchers not reporting negative results) or publication bias (journal rejects negative results)

 

Angus Hotchkies 10/05 08:27PM

ok thanks

 

Daniel Lewis 10/05 08:28PM

If we were to do this with say studies on an expensive new drug – the funnel plot would likely be one sided, because drug companies dont like reporting negative studies

 

Bradley Storey 10/05 08:29PM

right, ok

 

Angus Hotchkies 10/05 08:29PM

ahh very interesting

 

Daniel Lewis 10/05 08:29PM

Some interesting watching for a rainy day on funnel plots and bad science https://www.ted.com/talks/ben_goldacre_battling_bad_science/transcript?language=en

 

Bradley Storey 10/05 08:30PM

Hehe, will make a note

 

Angus Hotchkies 10/05 08:30PM

cheers

 

Daniel Lewis 10/05 08:31PM

Back to this paper, is the heterogenity in blood loss surprising?

 

Angus Hotchkies 10/05 08:31PM

no because there’s a range of surgeries being carried out?

 

Sarah Michael 10/05 08:31PM

i would have thought they would all show less blood loss for diathermy?

 

Bradley Storey 10/05 08:32PM

Not really as presumably different operations will have different blood loss depending on scalpel vs diathermy

and maybe even experience of the surgeon favouring one over other, not sure if that matters or already taken into account

 

Daniel Lewis 10/05 08:34PM

Exactly, so whilst they all show less blood loss (minus two studies) they’re looking at different operations. What did you think about the incision time and hsopital stay data?

 

Angus Hotchkies 10/05 08:35PM

unsure how to interpret incision time. For hospital stay data surely there are far too many variables to interpret reliably?

 

Bradley Storey 10/05 08:36PM

Agreed and thats probably why they don’t really comment on hospital stay?

Just a question for my understanding, why would incisional time in seconds matter?

 

Daniel Lewis 10/05 08:38PM

Its a good question, there are very few situation s where saving seconds on an incision would generally matter – hosing rupture AAA, rapidly expanding extradural maybe?

For daily practice though, the decreased blood loss and incision time needs to be balanced against the increased risk of seroma in this meta-analysis

 

Bradley Storey 10/05 08:41PM

Mm yeah I agree I think

 

Daniel Lewis 10/05 08:42PM

A few final points before we close, on the seroma dat the i2 value is quite low meaning that almost all the studies were homogenous in saying the risk of seroma was higher

Would this meta-analysis change anyones management?

 

Angus Hotchkies 10/05 08:45PM

Sorry as in the meta-analysis for the seroma or overall?

 

Bradley Storey 10/05 08:45PM

I doubt it given that I don’t think the values measured seem to be the most clinically relevant of things, especially if complication risk (seroma) higher. Also most surgeons probably stick to what they prefer and have experience in? Finally though I’m not that aware of every journal, if it was more important findings – even though a well researched study – wouldn’t it be published in a more recognised journal; I’m not familiar with the Journal of Surgical Research?

 

Angus Hotchkies 10/05 08:48PM

Overall, I think it’s great for students to read as it makes us think about what we might use for incision should we become surgeons. I agree with Bradley in that surgeons will likely use their experience primarily but further studies may change managment in the future and this paper does seem extremely thorough!

 

Daniel Lewis 10/05 08:50PM

I agree with you both, I think their statistical methods are very thorough and a nice example of how do a meta-analysis. Will it change practice, probably not. Might have done away with the higher infection risk claim from diathermy but otherwise icnision time and blood loss (?clinically significant) are unlikely to change individual surgical practice

 

Bradley Storey 10/05 08:52PM

Yeah, great thank you

 

Angus Hotchkies 10/05 08:52PM

Yeah it seems like a good example of systematic review and meta-analysis to refer to! Thanks for breaking it down and making it easier to understand!

 

Sarah Michael 10/05 08:53PM

yeah thank you!

 

Daniel Lewis 10/05 08:54PM

No worries, glad it was useful

 

Bradley Storey 10/05 08:54PM

It was thanks again!

 

Angus Hotchkies 10/05 08:55PM

Thank you very much!

 

Daniel Lewis 10/05 08:55PM

See you all the next one, I best go before my other half takes a scalpel to me for being late!

 

Angus Hotchkies 10/05 08:56PM

Sounds like you better – cheers for your time 

 

Angus Hotchkies 10/05 08:58PM

cheers guys – certificates sent to you both 

 

Sarah Michael 10/05 08:59PM

Thanks angus!!

96

Normal
0

false
false
false

EN-GB
X-NONE
X-NONE

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:”Table Normal”;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:””;
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:”Calibri”,sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-fareast-language:EN-US;}