You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 11:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Mon Aug 24, 2026 00:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3839270991061,85cyclictest0-21swapper/023:54:590
3839285991055,18cyclictest131rcu_preempt22:15:023
3839270991011,79cyclictest0-21swapper/000:06:450
3839270991011,61cyclictest0-21swapper/000:20:150
3839270999925,26cyclictest0-21swapper/022:05:180
383927099991,56cyclictest3923867-21ssh23:57:440
3839270999424,28cyclictest0-21swapper/000:26:520
3839270999424,28cyclictest0-21swapper/000:26:510
383927099931,67cyclictest0-21swapper/023:18:070
383927099931,60cyclictest3884346-21ssh22:32:530
3839270999223,34cyclictest3927174-21ssh00:05:010
383927099921,50cyclictest0-21swapper/021:25:430
383927099901,55cyclictest0-21swapper/020:50:010
3839270998920,59cyclictest16850irq/134-800000021:35:000
383927099882,52cyclictest3883082-21latency_hist22:30:030
383927099882,52cyclictest3883082-21latency_hist22:30:020
3839270998726,26cyclictest3849311-21sh21:18:100
3839270998521,37cyclictest3850384-21ssh21:20:070
383927099841,78cyclictest48-21kcompactd023:21:440
383927099812,52cyclictest3941008-21sh00:34:100
383927099811,47cyclictest3861857-21cat21:45:020
383927099791,50cyclictest0-21swapper/000:35:340
3839270997829,17cyclictest12-21ksoftirqd/023:00:340
3839270997618,41cyclictest16850irq/134-800000023:33:350
383927099761,70cyclictest0-21swapper/023:29:230
383927099761,39cyclictest3880322-21ssh22:24:280
383927099751,43cyclictest0-21swapper/021:57:020
383927099742,46cyclictest364-21snmpd23:13:460
3839270997424,17cyclictest0-21swapper/022:18:320
3839270997424,17cyclictest0-21swapper/022:18:310
3839270997421,49cyclictest3893636-21rm22:52:370
3839270997420,28cyclictest12-21ksoftirqd/023:41:080
383927099732,39cyclictest3919259-21sh23:48:000
383927099732,39cyclictest3919259-21sh23:48:000
383927099731,40cyclictest0-21swapper/000:13:250
3839283997210,46cyclictest3901836-21ssh23:10:022
3839270997225,43cyclictest16950irq/224-800000020:55:010
3839270997221,22cyclictest12-21ksoftirqd/021:51:330
383927099721,44cyclictest0-21swapper/022:58:590
383927099721,39cyclictest3888577-21ssh22:41:350
3839270997127,27cyclictest3875035-21ssh22:13:060
383927099712,64cyclictest3885656-21ssh22:35:280
383927099712,40cyclictest3892350-21munin-run22:50:020
383927099711,61cyclictest0-21swapper/000:19:440
383927099711,61cyclictest0-21swapper/000:19:440
3839270997030,36cyclictest3900038-21ssh23:06:040
3839270996854,9cyclictest16950irq/224-800000019:20:000
3839270996844,18cyclictest12-21ksoftirqd/023:40:000
3839270996844,18cyclictest12-21ksoftirqd/023:40:000
3839270996755,9cyclictest16850irq/134-800000021:10:030
3839270996727,23cyclictest3858879-21ssh21:38:480
3839283996555,7cyclictest30-21ksoftirqd/221:45:012
3839283996554,7cyclictest3844196-21kworker/2:223:45:302
3839283996554,7cyclictest3844196-21kworker/2:223:45:292
383927099652,38cyclictest3843375-21latency_hist20:30:010
3839270996523,38cyclictest3847655-21sh21:14:400
383927099642,56cyclictest3842081-21munin-run20:05:000
383927099622,56cyclictest3844159-21latency_hist20:45:020
3839270996215,27cyclictest16850irq/134-800000022:01:080
383927099621,48cyclictest0-21swapper/021:49:190
383927099602,54cyclictest3843852-21sshd20:36:530
383927099602,54cyclictest3843852-21sshd20:36:530
383927099602,53cyclictest3841550-21munin-run19:55:000
383927099602,49cyclictest3844906-21cron21:00:000
383927099601,30cyclictest0-21swapper/019:50:000
3839283995938,15cyclictest1-21systemd21:55:002
3839278995914,6cyclictest131rcu_preempt23:20:011
383927099591,53cyclictest48-21kcompactd019:43:480
383927099591,53cyclictest48-21kcompactd019:43:480
383927099591,53cyclictest3843618-21cron20:35:010
3839283995849,6cyclictest3844196-21kworker/2:223:28:302
3839270995835,19cyclictest3845238-21cat21:05:010
3839270995828,23cyclictest275-21dbus-daemon20:15:010
3839285995730,18cyclictest37-21ksoftirqd/300:18:133
3839285995730,18cyclictest37-21ksoftirqd/300:18:123
3839283995749,5cyclictest3882953-21kworker/2:123:01:192
3839283995748,6cyclictest3828138-21kworker/2:021:25:542
3839283995746,6cyclictest3828138-21kworker/2:020:15:002
3839283995732,5cyclictest3828138-21kworker/2:021:14:232
383927099573,48cyclictest3839996-21cron19:25:010
383927099571,51cyclictest3840226-21sshd19:25:140
3839285995633,8cyclictest37-21ksoftirqd/322:07:183
3839283995647,5cyclictest3844196-21kworker/2:223:32:302
3839283995646,7cyclictest3882953-21kworker/2:123:24:342
3839283995642,5cyclictest3844196-21kworker/2:200:31:482
3839283995611,10cyclictest131rcu_preempt23:55:002
383927099563,50cyclictest3839466-21munin-run19:15:000
383927099562,49cyclictest364-21snmpd19:36:390
383927099562,48cyclictest364-21snmpd19:31:300
383927099561,52cyclictest48-21kcompactd020:16:320
3839285995525,9cyclictest37-21ksoftirqd/323:05:003
3839285995521,11cyclictest37-21ksoftirqd/322:00:023
3839285995510,5cyclictest131rcu_preempt22:30:013
3839285995510,5cyclictest131rcu_preempt22:30:003
3839283995547,5cyclictest3844196-21kworker/2:222:50:032
3839283995547,5cyclictest3844196-21kworker/2:200:27:582
3839283995547,5cyclictest3844196-21kworker/2:200:27:572
3839283995547,5cyclictest3828138-21kworker/2:021:55:142
3839283995546,6cyclictest3844196-21kworker/2:222:53:182
383927899556,9cyclictest23-21ksoftirqd/100:15:121
383927899556,9cyclictest23-21ksoftirqd/100:15:111
3839278995515,9cyclictest131rcu_preempt23:53:481
3839278995514,9cyclictest23-21ksoftirqd/121:11:141
3839278995512,14cyclictest23-21ksoftirqd/122:49:281
383927099551,50cyclictest0-21swapper/020:10:020
383927099551,49cyclictest3841777-21sshd19:57:140
383928599545,12cyclictest37-21ksoftirqd/323:40:013
383928599545,12cyclictest37-21ksoftirqd/323:40:003
3839285995415,6cyclictest131rcu_preempt23:53:333
3839283995445,6cyclictest3844196-21kworker/2:222:43:152
3839283995445,5cyclictest3828138-21kworker/2:020:40:012
3839283995445,5cyclictest3828138-21kworker/2:020:40:002
383927899548,5cyclictest131rcu_preempt22:55:021
3839278995442,8cyclictest221rcuc/122:10:291
383927899542,9cyclictest23-21ksoftirqd/100:02:181
3839278995414,6cyclictest131rcu_preempt00:00:021
3839278995413,7cyclictest131rcu_preempt22:00:011
3839278995412,10cyclictest23-21ksoftirqd/123:37:061
3839278995412,10cyclictest23-21ksoftirqd/123:37:061
3839278995410,6cyclictest131rcu_preempt23:05:011
383928599533,9cyclictest37-21ksoftirqd/321:55:013
3839285995317,11cyclictest37-21ksoftirqd/321:24:323
3839285995311,6cyclictest131rcu_preempt21:20:003
3839283995345,5cyclictest3882953-21kworker/2:123:00:012
3839283995345,5cyclictest3828138-21kworker/2:022:22:432
3839283995344,6cyclictest3828138-21kworker/2:021:38:032
3839283995344,5cyclictest3844196-21kworker/2:222:34:332
3839283995343,6cyclictest3844196-21kworker/2:223:41:212
3839283995342,6cyclictest3844196-21kworker/2:200:25:002
3839283995337,8cyclictest3844196-21kworker/2:223:39:032
3839283995337,8cyclictest3844196-21kworker/2:223:39:032
383927899538,5cyclictest131rcu_preempt23:12:521
383927899533,11cyclictest23-21ksoftirqd/121:30:021
3839278995331,16cyclictest275-21dbus-daemon23:10:011
3839278995313,6cyclictest131rcu_preempt23:50:011
3839278995313,6cyclictest131rcu_preempt23:50:011
3839278995311,5cyclictest131rcu_preempt21:21:251
383927099534,44cyclictest0-21swapper/019:10:030
383927099532,46cyclictest364-21snmpd20:22:480
383927099532,46cyclictest364-21snmpd20:22:470
383928599528,6cyclictest131rcu_preempt22:50:023
383928599524,8cyclictest37-21ksoftirqd/321:11:243
3839285995224,17cyclictest37-21ksoftirqd/323:28:513
3839285995211,7cyclictest131rcu_preempt00:05:013
3839285995211,6cyclictest131rcu_preempt00:09:453
3839283995244,5cyclictest3882953-21kworker/2:123:17:422
3839283995244,5cyclictest3828138-21kworker/2:022:01:462
3839283995244,5cyclictest3828138-21kworker/2:021:33:562
3839283995244,5cyclictest3828138-21kworker/2:020:55:022
3839283995243,6cyclictest3828138-21kworker/2:022:19:242
3839283995243,6cyclictest3828138-21kworker/2:022:19:242
3839283995212,10cyclictest131rcu_preempt00:20:012
3839283995212,10cyclictest131rcu_preempt00:20:002
383927899529,5cyclictest131rcu_preempt21:20:001
3839278995223,8cyclictest23-21ksoftirqd/123:28:151
3839278995212,6cyclictest131rcu_preempt23:43:471
383928599519,5cyclictest131rcu_preempt23:56:013
383928599516,7cyclictest37-21ksoftirqd/300:33:333
3839285995124,17cyclictest37-21ksoftirqd/319:50:013
383928599511,24cyclictest0-21swapper/322:01:593
3839283995143,5cyclictest3882953-21kworker/2:123:13:242
3839283995143,5cyclictest3844196-21kworker/2:223:58:362
383927899517,6cyclictest131rcu_preempt23:31:521
383927899516,6cyclictest131rcu_preempt21:50:391
383927899513,7cyclictest131rcu_preempt22:56:321
3839278995113,6cyclictest131rcu_preempt00:37:531
3839278995112,10cyclictest23-21ksoftirqd/122:29:501
3839278995112,10cyclictest23-21ksoftirqd/122:29:491
3839278995110,6cyclictest131rcu_preempt21:40:071
383928599508,6cyclictest131rcu_preempt21:39:423
383928599503,5cyclictest131rcu_preempt23:50:013
383928599503,5cyclictest131rcu_preempt23:50:003
3839285995033,13cyclictest275-21dbus-daemon23:35:023
3839285995012,9cyclictest131rcu_preempt00:21:243
3839285995011,6cyclictest131rcu_preempt23:44:593
3839283995042,5cyclictest3828138-21kworker/2:022:10:082
3839283995042,5cyclictest3828138-21kworker/2:022:08:562
3839283995042,5cyclictest3828138-21kworker/2:021:16:182
3839283995042,5cyclictest3828138-21kworker/2:020:35:012
3839283995042,5cyclictest3828138-21kworker/2:019:50:012
3839283995041,6cyclictest3844196-21kworker/2:200:04:402
3839283995041,6cyclictest3828138-21kworker/2:022:25:422
3839283995041,6cyclictest3828138-21kworker/2:022:25:422
3839283995036,9cyclictest30-21ksoftirqd/200:11:402
383927899507,6cyclictest23-21ksoftirqd/122:01:041
383927899503,6cyclictest131rcu_preempt00:31:501
383927899503,11cyclictest23-21ksoftirqd/121:35:241
3839278995027,8cyclictest23-21ksoftirqd/122:07:561
383927899502,5cyclictest131rcu_preempt23:24:231
3839278995013,5cyclictest131rcu_preempt21:49:411
383927899501,22cyclictest0-21swapper/121:04:131
3839278995011,6cyclictest131rcu_preempt00:22:451
383928599498,7cyclictest131rcu_preempt22:45:003
383928599498,6cyclictest131rcu_preempt21:00:013
383928599493,6cyclictest37-21ksoftirqd/322:17:573
383928599493,6cyclictest37-21ksoftirqd/322:17:573
3839285994919,21cyclictest37-21ksoftirqd/320:27:583
3839285994910,6cyclictest131rcu_preempt23:10:023
3839285994910,6cyclictest131rcu_preempt23:00:013
3839283994941,5cyclictest3844196-21kworker/2:200:35:492
3839283994941,5cyclictest3828138-21kworker/2:020:45:012
3839283994941,5cyclictest3828138-21kworker/2:019:35:032
3839283994939,6cyclictest3844196-21kworker/2:222:37:302
3839283994931,11cyclictest275-21dbus-daemon20:10:022
383927899498,5cyclictest131rcu_preempt00:10:191
383927899493,5cyclictest131rcu_preempt21:31:561
3839278994914,5cyclictest131rcu_preempt22:20:411
3839278994912,6cyclictest131rcu_preempt22:35:011
3839278994910,6cyclictest131rcu_preempt22:16:301
3839278994910,6cyclictest131rcu_preempt22:16:291
383928599489,6cyclictest131rcu_preempt19:25:023
383928599484,12cyclictest37-21ksoftirqd/321:34:183
3839285994812,5cyclictest131rcu_preempt19:30:013
383928399488,6cyclictest131rcu_preempt21:00:002
3839283994840,5cyclictest3844196-21kworker/2:221:47:192
3839283994839,6cyclictest3844196-21kworker/2:200:05:552
383927899489,5cyclictest131rcu_preempt22:42:441
383927899489,10cyclictest23-21ksoftirqd/119:53:141
383927899486,4cyclictest131rcu_preempt22:36:051
3839278994814,7cyclictest131rcu_preempt00:05:431
3839278994813,7cyclictest131rcu_preempt19:10:021
3839278994810,6cyclictest131rcu_preempt21:10:011
3839278994810,6cyclictest131rcu_preempt20:40:011
3839278994810,6cyclictest131rcu_preempt20:40:001
383928599479,6cyclictest131rcu_preempt22:24:593
3839285994733,11cyclictest3859714-21sshd21:40:143
3839285994716,8cyclictest37-21ksoftirqd/300:30:033
3839285994716,8cyclictest37-21ksoftirqd/300:30:033
383928599471,5cyclictest131rcu_preempt21:26:213
3839285994710,6cyclictest131rcu_preempt23:22:103
3839285994710,5cyclictest131rcu_preempt22:35:573
3839283994740,4cyclictest3828138-21kworker/2:020:25:042
3839283994738,5cyclictest3828138-21kworker/2:021:21:512
3839283994738,5cyclictest3828138-21kworker/2:020:48:142
383928599468,4cyclictest131rcu_preempt19:15:143
383928599466,10cyclictest37-21ksoftirqd/300:37:183
383928599465,11cyclictest37-21ksoftirqd/319:52:143
3839285994631,11cyclictest3840807-21pluginstate19:40:023
3839285994620,16cyclictest37-21ksoftirqd/322:51:543
3839285994611,5cyclictest131rcu_preempt22:30:363
3839283994638,5cyclictest3828138-21kworker/2:019:17:292
383927899469,6cyclictest131rcu_preempt00:25:251
383927899469,6cyclictest131rcu_preempt00:25:241
383927899466,4cyclictest131rcu_preempt20:55:141
383927899464,6cyclictest23-21ksoftirqd/120:17:151
3839278994611,5cyclictest131rcu_preempt19:25:021
383928599458,5cyclictest131rcu_preempt00:11:383
3839285994531,11cyclictest37-21ksoftirqd/323:10:533
383928599452,5cyclictest131rcu_preempt23:15:073
3839283994537,5cyclictest3828138-21kworker/2:019:40:022
383927899452,8cyclictest23-21ksoftirqd/120:00:001
3839278994512,5cyclictest131rcu_preempt19:30:011
383928599449,5cyclictest131rcu_preempt21:45:473
383928599446,10cyclictest37-21ksoftirqd/320:06:483
383928599445,6cyclictest131rcu_preempt20:20:003
3839285994430,11cyclictest3840590-21cut19:35:023
383928599442,8cyclictest37-21ksoftirqd/320:00:003
3839283994436,5cyclictest3828138-21kworker/2:020:03:282
3839278994436,6cyclictest23-21ksoftirqd/119:45:021
3839278994436,6cyclictest23-21ksoftirqd/119:45:021
383927899441,9cyclictest23-21ksoftirqd/119:35:001
383927899441,6cyclictest0-21swapper/119:36:141
3839285994313,7cyclictest361rcuc/320:35:013
383928599431,14cyclictest37-21ksoftirqd/320:54:133
3839283994335,5cyclictest3828138-21kworker/2:021:06:102
383927899435,5cyclictest131rcu_preempt20:05:021
3839278994327,11cyclictest309-21in:imuxsock20:45:001
383928599428,7cyclictest37-21ksoftirqd/321:10:013
383928599425,6cyclictest131rcu_preempt20:40:023
383928599425,6cyclictest131rcu_preempt20:40:023
3839285994227,11cyclictest1-21systemd20:25:023
3839285994227,11cyclictest1-21systemd20:25:013
3839285994222,17cyclictest37-21ksoftirqd/319:10:103
3839285994218,6cyclictest131rcu_preempt20:15:023
3839283994234,5cyclictest3828138-21kworker/2:019:10:172
383927899426,5cyclictest131rcu_preempt20:35:001
3839285994123,11cyclictest275-21dbus-daemon19:45:013
3839285994123,11cyclictest275-21dbus-daemon19:45:013
3839283994133,5cyclictest3828138-21kworker/2:020:20:022
3839283994133,5cyclictest3828138-21kworker/2:019:54:212
3839278994127,11cyclictest3467602-21systemd-journal20:45:331
383927899411,5cyclictest131rcu_preempt19:17:001
3839285994027,9cyclictest309-21in:imuxsock20:43:143
3839283994032,5cyclictest3828138-21kworker/2:019:25:522
383927899404,8cyclictest23-21ksoftirqd/120:55:001
3839283993926,9cyclictest309-21in:imuxsock20:23:152
3839283993926,9cyclictest309-21in:imuxsock20:23:142
383927899399,6cyclictest131rcu_preempt19:14:211
383927899391,7cyclictest23-21ksoftirqd/119:45:071
3839285993828,7cyclictest37-21ksoftirqd/319:10:023
3839283993831,4cyclictest3828138-21kworker/2:021:03:202
3839278993830,6cyclictest23-21ksoftirqd/120:29:591
3839278993811,5cyclictest131rcu_preempt20:21:091
3839278993811,5cyclictest131rcu_preempt20:21:091
383928599373,9cyclictest37-21ksoftirqd/321:04:133
3839285993714,6cyclictest131rcu_preempt20:05:023
383927899379,5cyclictest131rcu_preempt20:11:451
383927899375,8cyclictest23-21ksoftirqd/120:05:291
383928599364,7cyclictest37-21ksoftirqd/320:45:163
3839283993621,5cyclictest131rcu_preempt19:24:142
3839283993528,4cyclictest3828138-21kworker/2:019:57:312
3839283993015,5cyclictest131rcu_preempt19:40:552
3839283993015,5cyclictest131rcu_preempt19:40:542
3839283992720,4cyclictest3839208-21latency_hist19:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional