You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 23:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Mar 14, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
427299250,24cyclictest30443-21diskmemload12:30:233
427299250,24cyclictest17251-21sh10:30:093
427299250,24cyclictest10469-21pluginstate09:52:073
426899250,1cyclictest31143-21sh10:42:532
426899250,1cyclictest28065-21sh10:09:212
426599250,24cyclictest30443-21diskmemload10:53:241
426599250,24cyclictest17498-21ssh09:59:051
426599250,24cyclictest0-21swapper/111:38:401
426599250,24cyclictest0-21swapper/111:24:301
426599250,24cyclictest0-21swapper/110:57:581
4261992524,1cyclictest62550irq/127-eno111:17:430
426199250,24cyclictest62550irq/127-eno111:07:210
426199250,24cyclictest30443-21diskmemload11:41:580
426199250,24cyclictest0-21swapper/009:50:450
4272992422,1cyclictest3520-21ssh09:15:083
4272992421,2cyclictest9012-21sh10:52:413
427299240,23cyclictest0-21swapper/311:46:583
427299240,23cyclictest0-21swapper/310:06:363
427299240,23cyclictest0-21swapper/309:11:283
427299240,0cyclictest0-21swapper/309:57:073
4268992423,1cyclictest17780-21sh10:30:572
4268992422,1cyclictest30443-21diskmemload11:22:022
426899240,23cyclictest0-21swapper/209:47:542
426599240,23cyclictest30443-21diskmemload09:10:231
426599240,23cyclictest21861-21basename12:07:011
426599240,23cyclictest0-21swapper/112:10:291
426599240,0cyclictest11107-21nvmesmart_nvme012:27:061
4261992422,1cyclictest0-21swapper/012:10:410
4261992422,1cyclictest0-21swapper/009:16:160
4261992421,2cyclictest6931-21ssh12:22:360
4261992421,2cyclictest28549-21wc10:10:000
4261992421,2cyclictest0-21swapper/010:55:170
426199240,24cyclictest0-21swapper/010:15:280
426199240,1cyclictest0-21swapper/010:45:150
426199240,0cyclictest0-21swapper/010:10:480
4272992323,0cyclictest30443-21diskmemload11:39:373
4272992323,0cyclictest0-21swapper/312:02:573
4272992323,0cyclictest0-21swapper/311:58:593
4272992323,0cyclictest0-21swapper/310:10:193
4272992323,0cyclictest0-21swapper/309:39:363
4272992320,2cyclictest23894-21sh09:34:463
4272992320,2cyclictest10588-21sh10:23:233
427299230,23cyclictest0-21swapper/312:10:193
427299230,23cyclictest0-21swapper/311:43:583
427299230,23cyclictest0-21swapper/311:27:353
427299230,22cyclictest29221-21munin-run08:00:003
427299230,22cyclictest2570-21ssh10:16:373
427299230,22cyclictest25571-21rm10:37:183
427299230,22cyclictest25315-21sh11:08:443
427299230,22cyclictest24811-21apt-get07:51:583
427299230,22cyclictest20325-21ssh12:36:583
427299230,22cyclictest0-21swapper/311:52:093
427299230,22cyclictest0-21swapper/307:11:593
427299230,0cyclictest0-21swapper/312:06:313
427299230,0cyclictest0-21swapper/311:10:553
427299230,0cyclictest0-21swapper/309:22:023
4268992323,0cyclictest30443-21diskmemload12:31:532
4268992323,0cyclictest30443-21diskmemload12:20:112
4268992323,0cyclictest0-21swapper/212:03:482
4268992321,1cyclictest0-21swapper/212:35:392
4268992321,1cyclictest0-21swapper/209:41:472
4268992321,1cyclictest0-21swapper/209:23:082
4268992320,2cyclictest9146-21sh10:52:532
4268992320,2cyclictest8431-21grep09:20:012
426899231,21cyclictest313-21systemd-journal08:40:002
426899231,21cyclictest30443-21diskmemload10:22:012
426899230,23cyclictest30443-21diskmemload09:31:242
426899230,23cyclictest0-21swapper/211:49:572
426899230,23cyclictest0-21swapper/210:16:072
426899230,23cyclictest0-21swapper/209:39:092
426899230,22cyclictest9696-21df_inode09:52:002
426899230,22cyclictest9660-21ssh11:55:212
426899230,22cyclictest533-21ssh10:45:252
426899230,22cyclictest31231-21ssh12:15:512
426899230,22cyclictest21449-21ssh11:05:272
426899230,22cyclictest0-21swapper/211:35:492
426899230,22cyclictest0-21swapper/209:56:412
426899230,0cyclictest30443-21diskmemload12:06:152
426899230,0cyclictest0-21swapper/212:25:062
4265992323,0cyclictest0-21swapper/112:35:381
4265992323,0cyclictest0-21swapper/112:33:071
4265992323,0cyclictest0-21swapper/111:05:031
4265992323,0cyclictest0-21swapper/110:35:141
4265992323,0cyclictest0-21swapper/110:17:191
4265992323,0cyclictest0-21swapper/109:54:011
4265992323,0cyclictest0-21swapper/109:31:471
4265992323,0cyclictest0-21swapper/109:15:471
4265992321,1cyclictest9898-21diskmemload10:22:131
4265992321,1cyclictest3456-21sh11:18:511
4265992321,1cyclictest25671-21perf09:00:011
4265992321,1cyclictest0-21swapper/111:26:481
4265992321,1cyclictest0-21swapper/110:25:421
4265992320,2cyclictest28524-21ssh10:41:361
426599230,23cyclictest0-21swapper/111:57:231
426599230,23cyclictest0-21swapper/111:46:291
426599230,22cyclictest4455-21sh12:21:321
426599230,22cyclictest313-21systemd-journal07:20:011
426599230,22cyclictest24399-21apt07:50:001
426599230,22cyclictest14491-21ssh09:25:561
426599230,22cyclictest0-21swapper/112:15:321
426599230,22cyclictest0-21swapper/110:05:331
426599230,0cyclictest30443-21diskmemload11:31:281
426599230,0cyclictest0-21swapper/111:44:371
426599230,0cyclictest0-21swapper/110:11:261
4261992323,0cyclictest0-21swapper/011:50:510
4261992323,0cyclictest0-21swapper/009:35:300
4261992322,1cyclictest0-21swapper/011:47:070
4261992321,1cyclictest0-21swapper/010:51:530
4261992321,1cyclictest0-21swapper/009:22:210
426199230,2cyclictest0-21swapper/009:40:560
426199230,23cyclictest0-21swapper/012:26:400
426199230,23cyclictest0-21swapper/009:56:350
426199230,23cyclictest0-21swapper/009:25:570
426199230,22cyclictest9744-21diskmemload11:55:290
426199230,22cyclictest62550irq/127-eno111:11:000
426199230,22cyclictest606-21dbus-daemon09:15:010
426199230,22cyclictest30443-21diskmemload10:23:090
426199230,22cyclictest28074-21turbostat.cron09:05:010
426199230,22cyclictest18170-21sh12:33:520
426199230,22cyclictest15787-21sh10:28:130
426199230,21cyclictest0-21swapper/010:31:250
426199230,0cyclictest30443-21diskmemload11:04:500
426199230,0cyclictest0-21swapper/012:01:590
426199230,0cyclictest0-21swapper/009:33:030
4272992222,0cyclictest0-21swapper/311:30:073
4272992222,0cyclictest0-21swapper/310:45:423
4272992222,0cyclictest0-21swapper/309:25:203
4272992221,1cyclictest0-21swapper/312:15:253
4272992220,1cyclictest16298-21grep07:35:013
4272992220,1cyclictest0-21swapper/311:20:023
4272992220,1cyclictest0-21swapper/310:56:133
4272992220,1cyclictest0-21swapper/310:00:063
427299220,22cyclictest0-21swapper/312:25:233
427299220,22cyclictest0-21swapper/311:15:563
427299220,21cyclictest18480-21munin-run08:45:003
427299220,21cyclictest0-21swapper/312:23:453
427299220,21cyclictest0-21swapper/309:46:113
427299220,21cyclictest0-21swapper/307:25:013
427299220,1cyclictest0-21swapper/310:41:203
427299220,1cyclictest0-21swapper/310:26:203
4268992222,0cyclictest0-21swapper/211:50:382
4268992222,0cyclictest0-21swapper/210:57:432
4268992221,1cyclictest25614-21ssh12:10:462
4268992219,2cyclictest11035-21ssh11:25:502
426899220,22cyclictest0-21swapper/210:12:372
426899220,21cyclictest30443-21diskmemload11:00:272
426899220,21cyclictest26893-21ssh11:41:042
426899220,21cyclictest26631-21ssh11:10:172
426899220,21cyclictest16294-21apt-get07:35:022
426899220,1cyclictest4403-21apt-get08:16:592
426899220,1cyclictest0-21swapper/211:31:532
426899220,1cyclictest0-21swapper/210:35:242
426899220,0cyclictest30443-21diskmemload10:25:192
4265992222,0cyclictest30443-21diskmemload11:10:161
4265992222,0cyclictest0-21swapper/111:00:091
4265992222,0cyclictest0-21swapper/109:45:041
4265992222,0cyclictest0-21swapper/109:20:341
4265992220,1cyclictest0-21swapper/110:31:261
426599220,22cyclictest0-21swapper/112:03:591
426599220,22cyclictest0-21swapper/109:41:341
426599220,21cyclictest32019-21apt-get08:06:591
426599220,21cyclictest0-21swapper/111:50:041
426599220,1cyclictest6416-21turbostat08:20:011
426599220,1cyclictest24443-21sh09:35:121
426599220,1cyclictest19181-21ssh10:01:101
426599220,1cyclictest0-21swapper/110:47:041
426199222,20cyclictest62550irq/127-eno110:40:590
4261992222,0cyclictest0-21swapper/012:35:040
4261992221,1cyclictest3313-21sh09:46:260
4261992221,0cyclictest17104-21apt-get07:36:590
4261992220,2cyclictest20217-21diskmemload12:05:360
4261992220,2cyclictest0-21swapper/011:22:270
4261992220,1cyclictest29230-21awk08:00:000
426199220,22cyclictest0-21swapper/011:35:380
426199220,22cyclictest0-21swapper/011:31:580
426199220,21cyclictest30046-21apt-get12:15:020
426199220,21cyclictest0-21swapper/008:56:590
426199220,21cyclictest0-21swapper/007:21:580
426199220,1cyclictest0-21swapper/010:00:050
426199220,0cyclictest0-21swapper/011:25:070
427299210,21cyclictest0-21swapper/311:00:123
427299210,20cyclictest0-21swapper/308:51:583
427299210,20cyclictest0-21swapper/308:36:583
427299210,0cyclictest0-21swapper/308:31:593
427299210,0cyclictest0-21swapper/308:21:593
4268992120,1cyclictest2001-21apt-get08:11:592
4268992119,1cyclictest22234-21apt-get07:45:022
426899210,20cyclictest30443-21diskmemload09:25:532
4265992120,0cyclictest21269-21apt-get08:51:581
4265992119,1cyclictest8822-21munin-run08:25:011
4261992121,0cyclictest0-21swapper/008:41:580
4261992121,0cyclictest0-21swapper/008:21:590
4261992119,1cyclictest0-21swapper/007:45:020
4261992118,2cyclictest19628-21apt07:41:590
426199210,21cyclictest0-21swapper/010:36:350
426199210,20cyclictest0-21swapper/008:46:580
426199210,0cyclictest0-21swapper/008:26:590
4272992020,0cyclictest0-21swapper/308:26:593
4272992018,1cyclictest3999-21perf08:15:013
4272992018,1cyclictest30464-21perf09:10:013
427299200,1cyclictest23666-21apt-get08:56:593
4268992020,0cyclictest28474-21apt-get09:06:592
4268992020,0cyclictest0-21swapper/208:06:592
4268992020,0cyclictest0-21swapper/208:01:582
4268992019,1cyclictest4655-21apt-get07:11:592
4268992017,2cyclictest9045-21dump-pmu-power07:20:012
426899200,20cyclictest0-21swapper/208:57:002
426899200,0cyclictest19629-21apt-get07:41:582
4265992020,0cyclictest0-21swapper/107:57:001
4265992019,0cyclictest0-21swapper/107:36:581
4265992018,2cyclictest0-21swapper/107:11:591
426599200,20cyclictest0-21swapper/108:41:581
4261992020,0cyclictest0-21swapper/009:06:590
4272991919,0cyclictest0-21swapper/308:16:593
4272991918,0cyclictest0-21swapper/307:16:593
4268991919,0cyclictest18872-21apt-get08:46:592
4268991919,0cyclictest0-21swapper/207:51:582
4265991919,0cyclictest0-21swapper/109:06:581
4265991919,0cyclictest0-21swapper/108:46:581
4265991919,0cyclictest0-21swapper/108:31:591
4265991919,0cyclictest0-21swapper/108:11:591
4265991919,0cyclictest0-21swapper/107:51:571
426599190,19cyclictest0-21swapper/107:41:591
4261991919,0cyclictest0-21swapper/007:51:580
426199190,18cyclictest4655-21apt-get07:11:590
4272991818,0cyclictest0-21swapper/307:41:583
4272991818,0cyclictest0-21swapper/307:35:023
4272991818,0cyclictest0-21swapper/307:27:003
4272991816,1cyclictest20847-21sh08:50:003
4268991818,0cyclictest0-21swapper/208:41:582
4268991817,1cyclictest6822-21apt-get08:21:592
4265991818,0cyclictest0-21swapper/107:27:001
4265991816,1cyclictest11222-21awk08:30:011
4261991818,0cyclictest0-21swapper/008:01:580
4261991818,0cyclictest0-21swapper/007:16:590
4261991816,1cyclictest0-21swapper/008:15:010
4272991717,0cyclictest0-21swapper/307:47:003
4272991716,0cyclictest0-21swapper/309:01:583
4268991717,0cyclictest27210-21apt-get07:56:592
4268991717,0cyclictest11614-21apt-get08:31:592
4265991717,0cyclictest9470-21apt-get07:21:581
4265991715,1cyclictest29623-21apt-get08:01:581
426899160,15cyclictest0-21swapper/207:32:002
4261991614,1cyclictest0-21swapper/008:10:010
426199160,15cyclictest15468-21switchtime08:37:060
427299150,14cyclictest0-21swapper/308:08:323
426899150,5cyclictest0-21swapper/207:21:582
427299140,14cyclictest0-21swapper/308:00:263
426899140,14cyclictest0-21swapper/208:50:202
426899140,14cyclictest0-21swapper/208:25:412
426899140,14cyclictest0-21swapper/207:26:492
426899140,0cyclictest0-21swapper/209:00:102
426599140,14cyclictest0-21swapper/107:30:501
426599140,0cyclictest0-21swapper/108:36:041
4261991414,0cyclictest0-21swapper/008:50:590
4261991414,0cyclictest0-21swapper/008:30:400
4261991414,0cyclictest0-21swapper/008:15:150
426199140,14cyclictest0-21swapper/007:30:010
426199140,14cyclictest0-21swapper/007:25:560
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional