You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 18:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Thu Nov 13, 2025 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
882721450,1sleep00-21swapper/011:40:430
3265121440,0sleep132650-21sshd09:13:031
936821370,1sleep29363-21rm10:51:592
1473121270,1sleep30-21swapper/310:49:233
524221260,0sleep00-21swapper/009:33:260
2010621150,2sleep30-21swapper/311:51:413
1196929533,43sleep10-21swapper/107:09:481
153642930,4sleep21224899cyclictest09:30:552
131072930,3sleep20-21swapper/211:34:472
125222920,1sleep11224799cyclictest12:00:401
227762910,1sleep00-21swapper/010:59:540
85662880,2sleep31848-21nfsd09:26:563
249982880,2sleep324994-21bash11:13:093
235742870,2sleep30-21swapper/310:27:233
231812860,3sleep20-21swapper/211:16:132
218242860,3sleep30-21swapper/309:18:263
167412850,1sleep10-21swapper/110:55:591
157752850,3sleep30-21swapper/310:55:533
245532840,1sleep024555-21id09:41:470
92022830,2sleep39203-21sshd09:40:183
245882830,2sleep124576-21sshd10:30:421
116932830,1sleep30-21swapper/312:20:173
181512820,2sleep218147-21sshd10:07:142
101012820,2sleep30-21swapper/310:42:263
326852810,1sleep20-21swapper/211:46:182
263152810,1sleep31848-21nfsd11:29:433
129022810,2sleep20-21swapper/211:50:552
1173428132,45sleep30-21swapper/307:06:493
89202760,0sleep00-21swapper/010:42:190
62652760,0sleep00-21swapper/010:28:570
170442760,3sleep20-21swapper/209:14:442
173282750,1sleep017322-21sshd09:17:550
8992740,2sleep2872-21sshd12:25:262
33202740,3sleep31224999cyclictest10:15:373
30702730,0sleep00-21swapper/009:23:040
23362730,2sleep22326-21sshd11:20:372
119562730,1sleep211955-21bash11:28:182
45972710,4sleep31224999cyclictest12:16:303
26782700,2sleep10-21swapper/112:38:231
309542680,3sleep31224999cyclictest11:56:013
23042680,0sleep11846-21nfsd09:19:481
179882680,1sleep10-21swapper/112:07:541
305882660,1sleep20-21swapper/210:31:192
123862600,0sleep00-21swapper/011:25:030
1172225735,19sleep00-21swapper/007:06:420
117042546,11sleep20-21swapper/207:06:282
5522480,1sleep10-21swapper/112:28:371
12246994133,6cyclictest0-21swapper/011:59:370
12249993732,4cyclictest26908-21sshd12:12:023
12247993733,2cyclictest0-21swapper/112:12:021
113432370,35sleep2135650irq/27-eth012:00:342
218522350,2sleep30-21swapper/309:21:433
103322350,0sleep310331-21bash09:10:403
13812330,2sleep31848-21nfsd12:31:483
1224699329,9cyclictest0-21swapper/012:32:340
30032300,3sleep20-21swapper/209:16:242
12246993012,5cyclictest0-21swapper/011:17:390
12246993011,5cyclictest3-21ksoftirqd/012:16:560
35012290,0sleep00-21swapper/010:15:380
303382290,2sleep10-21swapper/109:58:361
163772290,26sleep2135650irq/27-eth010:13:412
1224899290,19cyclictest33-21ksoftirqd/211:44:472
126152280,2sleep30-21swapper/311:47:353
12247992722,3cyclictest29829-21munin-run09:00:001
52262260,23sleep2135650irq/27-eth011:14:282
237862260,3sleep323783-21sshd11:22:503
175682260,2sleep217565-21bash12:20:492
282892250,1sleep00-21swapper/007:45:120
12248992514,8cyclictest5229-21sshd11:37:092
320992240,0sleep041ktimersoftd/011:49:360
86682230,1sleep00-21swapper/010:02:590
44262231,20sleep2135650irq/27-eth008:01:132
12249992320,2cyclictest29828-21perf09:00:003
1224899239,6cyclictest36-21rcuop/211:05:092
188722221,19sleep1135650irq/27-eth011:12:311
156172220,2sleep115604-21sshd10:20:131
138342220,2sleep30-21swapper/309:56:573
12248992214,3cyclictest1847-21nfsd12:19:302
12248992212,5cyclictest28559-21sshd11:03:462
1224699221,12cyclictest0-21swapper/011:35:430
12246992211,10cyclictest15125-21sshd11:12:080
91932210,1sleep09194-21bash09:53:180
307472210,2sleep22405-21runrttasks10:18:282
302922210,2sleep10-21swapper/109:22:331
288532210,2sleep00-21swapper/012:09:010
12248992111,4cyclictest10059-21sshd12:07:042
1224899210,19cyclictest33-21ksoftirqd/211:59:082
12246992112,4cyclictest3-21ksoftirqd/011:08:080
1224699211,15cyclictest0-21swapper/011:30:530
12246992110,5cyclictest3-21ksoftirqd/011:01:060
245972200,0sleep224598-21sshd10:20:572
12248992013,6cyclictest29677-21sshd12:12:182
12246992011,3cyclictest0-21swapper/012:11:150
1224699201,11cyclictest0-21swapper/012:24:570
1224699200,2cyclictest0-21swapper/011:50:380
96752190,1sleep19674-21bash09:50:021
311712190,16sleep226950irq/26-ahci[00009:39:182
12248991914,1cyclictest9119-21sshd12:39:042
1224899191,2cyclictest0-21swapper/207:42:372
12248991910,7cyclictest27919-21sshd12:31:132
12247991914,3cyclictest17226-21bash11:54:421
12247991913,5cyclictest1848-21nfsd11:08:111
1224699193,5cyclictest24840-21sshd12:37:160
1224699190,15cyclictest0-21swapper/010:48:030
263032180,1sleep00-21swapper/009:28:490
12247991813,4cyclictest3952-21sshd12:22:401
12247991813,3cyclictest8225-21sshd11:18:031
12247991813,3cyclictest4678-21bash11:40:201
1224699180,9cyclictest1845-21nfsd11:24:540
126682170,15sleep226950irq/26-ahci[00009:53:412
1224999177,8cyclictest984-21sshd11:43:133
12249991712,4cyclictest1848-21nfsd11:00:483
12249991712,3cyclictest0-21swapper/311:08:113
12248991712,3cyclictest25947-21rm10:40:442
12247991713,2cyclictest0-21swapper/111:00:491
1224699179,5cyclictest0-21swapper/010:32:250
1224699178,1cyclictest0-21swapper/012:27:340
1224699170,12cyclictest7288-21sshd10:09:300
107992170,1sleep010796-21sshd10:13:050
229752160,2sleep20-21swapper/210:04:302
12249991614,1cyclictest0-21swapper/312:28:083
12249991612,3cyclictest1959-21sshd12:36:143
12249991612,3cyclictest0-21swapper/311:18:033
1224999161,14cyclictest26692-21sshd10:30:553
12249991611,4cyclictest0-21swapper/307:35:053
1224999161,12cyclictest1848-21nfsd11:32:593
1224999160,6cyclictest0-21swapper/310:13:463
1224899168,3cyclictest6678-21sshd10:48:342
1224899167,4cyclictest7073-21sshd09:26:472
12248991612,3cyclictest29841-21kworker/2:209:41:102
1224799161,14cyclictest4046-21sshd10:45:061
12247991611,4cyclictest15350-21sshd11:21:581
1224799161,14cyclictest144550irq/28-eth1-rx-08:38:251
1224799160,15cyclictest6061-21sshd11:37:141
12246991611,1cyclictest0-21swapper/009:55:500
12246991610,2cyclictest3-21ksoftirqd/012:00:150
1224699160,1cyclictest0-21swapper/009:48:130
1224699160,10cyclictest0-21swapper/010:37:460
317362151,12sleep2135650irq/27-eth009:00:472
276922150,13sleep2135650irq/27-eth009:22:192
177092150,8sleep03-21ksoftirqd/008:30:230
12249991511,3cyclictest20640-21sshd10:04:143
1224999150,7cyclictest0-21swapper/312:03:413
1224999150,14cyclictest18415-21sshd10:07:163
1224999150,14cyclictest0-21swapper/311:38:373
1224899152,7cyclictest135650irq/27-eth008:11:272
1224899152,2cyclictest0-21swapper/208:09:412
1224899152,12cyclictest18058-21sshd10:36:372
1224899151,5cyclictest14609-21sshd10:55:462
1224899151,3cyclictest18852-21bash09:57:272
12248991511,3cyclictest10950irq/23-ehci_hcd07:30:482
1224899150,14cyclictest12351-21sshd10:26:142
1224799153,3cyclictest0-21swapper/111:26:071
1224799153,11cyclictest31962-21sshd11:56:081
12247991511,3cyclictest1847-21nfsd10:05:021
12247991511,2cyclictest0-21swapper/110:35:171
1224799150,7cyclictest0-21swapper/109:28:461
1224799150,14cyclictest6135-21sshd09:40:021
1224799150,14cyclictest0-21swapper/110:28:041
1224799150,14cyclictest0-21swapper/109:31:361
1224799150,13cyclictest1927-21snmpd07:45:111
1224699152,8cyclictest18049-21munin-node10:20:200
12246991510,1cyclictest0-21swapper/009:12:210
1224699150,4cyclictest0-21swapper/009:35:160
1224699150,4cyclictest0-21swapper/008:20:290
1224699150,14cyclictest0-21swapper/010:50:220
1224999142,1cyclictest0-21swapper/308:38:253
12249991412,1cyclictest0-21swapper/310:22:203
1224999141,12cyclictest1848-21nfsd07:27:323
1224999141,12cyclictest1848-21nfsd07:27:323
1224999141,12cyclictest14125-21sshd09:53:493
1224999141,12cyclictest10250-21sshd10:35:493
1224999141,11cyclictest22360-21sshd12:05:073
1224999141,11cyclictest1927-21snmpd09:35:463
1224999140,13cyclictest1049-21sshd09:45:543
1224999140,13cyclictest0-21swapper/308:42:413
1224999140,13cyclictest0-21swapper/307:10:153
1224999140,12cyclictest20581-21sshd09:31:303
1224899143,10cyclictest10950irq/23-ehci_hcd08:15:512
1224899141,9cyclictest17645-21munin-node08:30:222
1224899141,5cyclictest0-21swapper/208:55:452
1224899140,13cyclictest29953-21runrttasks07:46:162
1224899140,12cyclictest24496-21sshd09:45:082
1224799148,4cyclictest29676-21sshd10:15:091
1224799147,1cyclictest10894-21sshd12:17:071
1224799145,8cyclictest0-21swapper/112:30:121
1224799142,11cyclictest6922-21sshd10:42:081
1224799141,4cyclictest6029-21bash11:34:041
1224799140,6cyclictest17683-21bash10:10:241
1224799140,13cyclictest11254-21sshd10:03:161
1224799140,13cyclictest0-21swapper/109:35:421
1224799140,13cyclictest0-21swapper/108:09:121
1224799140,13cyclictest0-21swapper/107:50:501
1224799140,12cyclictest9126-21sshd11:47:131
1224799140,12cyclictest23976-21munin-node10:50:141
1224799140,12cyclictest14146-21rm09:47:151
1224699145,2cyclictest0-21swapper/007:19:590
1224699143,10cyclictest0-21swapper/008:54:590
1224699142,11cyclictest0-21swapper/008:55:160
1224699140,12cyclictest0-21swapper/008:04:130
1224699140,12cyclictest0-21swapper/007:22:530
1224699140,11cyclictest0-21swapper/007:54:480
1224999137,5cyclictest0-21swapper/309:06:293
12249991311,1cyclictest0-21swapper/308:01:253
12249991310,2cyclictest0-21swapper/310:51:403
1224999130,5cyclictest0-21swapper/307:15:043
1224999130,4cyclictest790-21fschecks_time07:55:133
1224999130,12cyclictest0-21swapper/308:11:153
1224999130,12cyclictest0-21swapper/308:09:513
1224999130,12cyclictest0-21swapper/307:30:063
1224999130,12cyclictest0-21swapper/307:21:093
1224899138,4cyclictest26353-21chrt08:50:162
1224899137,3cyclictest10950irq/23-ehci_hcd08:48:092
1224899132,7cyclictest135650irq/27-eth007:12:192
1224899131,3cyclictest12951-21ntp_states08:20:182
1224899130,12cyclictest0-21swapper/209:07:422
1224899130,12cyclictest0-21swapper/208:40:282
1224899130,12cyclictest0-21swapper/208:25:422
1224899130,12cyclictest0-21swapper/207:37:352
1224899130,12cyclictest0-21swapper/207:28:562
1224899130,12cyclictest0-21swapper/207:28:562
1224899130,11cyclictest19796-21open_files08:35:202
1224799137,5cyclictest0-21swapper/107:26:401
1224799137,5cyclictest0-21swapper/107:26:401
1224799137,5cyclictest0-21swapper/107:20:421
1224799134,4cyclictest0-21swapper/107:55:211
1224799131,11cyclictest26706-21iostat_ios07:40:151
1224799130,5cyclictest0-21swapper/108:15:481
1224799130,5cyclictest0-21swapper/107:33:341
1224799130,4cyclictest2405-21runrttasks07:15:591
1224799130,4cyclictest2405-21runrttasks07:10:101
1224799130,12cyclictest144550irq/28-eth1-rx-08:33:381
1224799130,12cyclictest0-21swapper/108:10:081
1224799130,12cyclictest0-21swapper/108:01:021
1224799130,12cyclictest0-21swapper/107:36:041
1224799130,11cyclictest30367-21df_inode09:00:131
1224799130,11cyclictest14848-21iostat_ios08:25:171
1224699139,1cyclictest0-21swapper/007:31:110
1224699138,2cyclictest0-21swapper/009:05:480
1224699138,2cyclictest0-21swapper/008:45:360
1224699136,3cyclictest0-21swapper/008:37:060
1224699134,3cyclictest7026-21grep08:10:010
1224699130,1cyclictest20061-21needreboot07:25:150
1224699130,1cyclictest20061-21needreboot07:25:150
1224699130,12cyclictest0-21swapper/007:41:480
1224699130,11cyclictest0-21swapper/009:00:300
1224699130,10cyclictest0-21swapper/008:25:000
1224699130,10cyclictest0-21swapper/008:10:240
1224699130,10cyclictest0-21swapper/007:37:290
1224999129,2cyclictest0-21swapper/307:50:133
1224999128,3cyclictest2405-21runrttasks07:45:073
1224999128,3cyclictest12567-21iostat08:20:153
1224999121,10cyclictest0-21swapper/309:00:173
12249991210,1cyclictest0-21swapper/308:30:173
1224999120,11cyclictest26736-21kernelversion07:40:173
1224999120,11cyclictest1848-21nfsd08:25:123
1224999120,11cyclictest0-21swapper/308:50:433
1224999120,11cyclictest0-21swapper/308:45:333
1224999120,11cyclictest0-21swapper/308:16:573
1224899125,3cyclictest10950irq/23-ehci_hcd07:50:142
1224899120,11cyclictest0-21swapper/207:56:072
1224899120,10cyclictest17496-21munin-node07:20:142
1224899120,10cyclictest15222-21munin-node07:15:152
1224799120,3cyclictest626-21awk09:05:161
1224799120,3cyclictest25805-21diskstats08:50:121
1224799120,3cyclictest21684-21iostat08:40:171
1224799120,11cyclictest12124-21df_inode08:20:111
1224799120,10cyclictest23879-21hddtemp_smartct08:45:161
1224699122,9cyclictest1927-21snmpd07:55:280
12246991210,1cyclictest0-21swapper/008:40:580
1224699120,1cyclictest0-21swapper/007:14:580
1224699110,10cyclictest10140-21perl08:15:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional