You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-15 - 11:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Aug 15, 2026 00:45:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
893474991111,64cyclictest970430-21date23:40:020
893474999429,42cyclictest918474-21latency_hist21:50:030
893474999323,42cyclictest0-21swapper/023:46:170
893474998931,54cyclictest16950irq/224-800000023:05:010
893474998928,57cyclictest926898-21ssh22:07:340
89347499881,62cyclictest0-21swapper/022:32:010
89347499881,51cyclictest0-21swapper/000:05:000
89347499852,49cyclictest935490-21ssh22:25:240
89347499841,64cyclictest0-21swapper/023:59:170
89347499832,53cyclictest899654-21diskmemload00:35:520
89347499832,39cyclictest916221-21munin-run21:45:010
893474998319,55cyclictest16850irq/134-800000022:00:400
89347499831,54cyclictest951793-21cat23:00:010
89347499829,29cyclictest0-21swapper/021:13:080
893474998075,3cyclictest0-21swapper/000:15:010
89347499802,42cyclictest993587-21ssh00:29:000
89347499801,75cyclictest0-21swapper/000:07:190
89347499801,75cyclictest0-21swapper/000:07:190
893479997910,47cyclictest879595-21munin-node21:45:011
893474997927,24cyclictest12-21ksoftirqd/021:34:470
89347499783,42cyclictest956444-21ssh23:10:190
89347499782,45cyclictest894717-21munin-run19:35:010
89347499782,45cyclictest894717-21munin-run19:35:000
893474997715,34cyclictest909169-21latency_hist21:30:020
89347499771,36cyclictest933318-21ssh22:20:550
893474997529,30cyclictest947637-21ssh22:51:160
893474997525,45cyclictest16850irq/134-800000023:10:000
89347499742,44cyclictest968025-21kworker/u8:323:53:220
89347499742,40cyclictest923252-21systemctl22:00:010
89347499742,40cyclictest923252-21systemctl22:00:010
89347499742,38cyclictest990067-21ssh00:21:280
893474997338,32cyclictest16950irq/224-800000021:23:080
893474997229,25cyclictest931849-21ssh22:18:040
89347499721,57cyclictest150-21pr/ttyS222:43:460
89347499721,37cyclictest967726-21ssh23:34:200
89347499721,37cyclictest899128-21munin-run20:59:590
89347499713,36cyclictest944877-21ssh22:45:150
89347499711,66cyclictest0-21swapper/021:39:240
893474996930,23cyclictest970713-21ssh23:40:150
893474996911,34cyclictest930301-21ssh22:14:580
893474996832,32cyclictest961461-21sh23:20:490
893474996824,28cyclictest0-21swapper/023:17:220
893474996815,29cyclictest0-21swapper/000:15:070
893474996720,43cyclictest902426-21rm21:15:400
89347499671,62cyclictest0-21swapper/021:54:570
89347499671,38cyclictest895834-21latency_hist19:55:020
89347499651,34cyclictest963721-21ssh23:25:390
893483996439,8cyclictest925658-21kworker/2:323:00:182
893474996424,23cyclictest995344-21ssh00:32:420
89347499641,33cyclictest941429-21ssh22:38:130
89347499633,55cyclictest898881-21munin-run20:55:020
89347499633,55cyclictest898881-21munin-run20:55:020
89348699625,15cyclictest37-21ksoftirqd/323:20:003
89347499621,56cyclictest1-21systemd20:29:380
89347499611,38cyclictest893907-21sshd19:18:360
89347499611,38cyclictest893907-21sshd19:18:350
89347499602,53cyclictest896282-21munin-run20:04:590
89347499601,54cyclictest899629-21sshd21:08:410
89347499601,53cyclictest48-21kcompactd020:34:220
893486995950,6cyclictest361rcuc/300:35:263
893483995948,6cyclictest925658-21kworker/2:323:12:302
89347499592,53cyclictest894209-21munin-run19:25:000
89347499592,52cyclictest898627-21munin-run20:50:000
89347499591,53cyclictest48-21kcompactd019:41:060
89348699589,9cyclictest37-21ksoftirqd/323:44:023
89348699588,9cyclictest37-21ksoftirqd/300:24:453
89348699585,18cyclictest37-21ksoftirqd/300:20:003
893486995823,9cyclictest37-21ksoftirqd/323:02:493
893483995849,6cyclictest894489-21kworker/2:122:03:202
893474995847,8cyclictest12-21ksoftirqd/020:14:370
89347499582,52cyclictest898363-21munin-run20:45:010
89347499582,52cyclictest893670-21munin-run19:15:000
89347499582,51cyclictest895510-21munin-run19:50:010
89347499581,53cyclictest894948-21sshd19:39:360
89348699576,11cyclictest37-21ksoftirqd/300:14:363
89348399579,34cyclictest3467602-21systemd-journal23:20:032
89347499571,52cyclictest894432-21sshd19:26:510
893474995714,39cyclictest895980-21sshd19:58:370
893486995618,10cyclictest37-21ksoftirqd/321:46:043
893486995616,9cyclictest37-21ksoftirqd/323:15:013
893486995614,7cyclictest37-21ksoftirqd/321:05:023
893486995613,12cyclictest37-21ksoftirqd/322:41:593
893486995612,4cyclictest131rcu_preempt21:21:353
893483995646,7cyclictest925658-21kworker/2:323:32:292
893479995647,6cyclictest23-21ksoftirqd/122:50:011
89347499561,50cyclictest898062-21sshd20:38:370
893486995515,6cyclictest131rcu_preempt23:48:503
893483995546,6cyclictest925658-21kworker/2:323:49:342
893483995545,5cyclictest894489-21kworker/2:122:07:382
89347999551,8cyclictest982183-21munin-run00:05:001
89348699544,10cyclictest37-21ksoftirqd/323:30:513
893486995417,9cyclictest37-21ksoftirqd/322:47:293
893486995410,15cyclictest37-21ksoftirqd/322:07:123
893486995410,14cyclictest37-21ksoftirqd/321:54:263
893483995445,6cyclictest867038-21kworker/2:319:30:022
893483995445,5cyclictest894489-21kworker/2:121:51:412
893483995444,6cyclictest994046-21kworker/2:100:40:022
89347499541,49cyclictest1-21systemd20:25:010
893483995344,6cyclictest894489-21kworker/2:121:35:072
893483995344,5cyclictest925658-21kworker/2:323:45:012
893483995344,5cyclictest867038-21kworker/2:321:05:022
89347999539,5cyclictest131rcu_preempt21:21:411
89347999537,8cyclictest23-21ksoftirqd/100:20:001
893479995331,17cyclictest3467602-21systemd-journal22:55:001
89348699527,7cyclictest37-21ksoftirqd/322:29:543
89348699526,9cyclictest37-21ksoftirqd/300:33:143
89348699524,8cyclictest37-21ksoftirqd/323:54:443
893486995218,9cyclictest37-21ksoftirqd/300:03:173
893483995245,4cyclictest958508-21kworker/2:200:00:032
893483995243,5cyclictest894489-21kworker/2:122:20:012
893483995243,5cyclictest894489-21kworker/2:121:18:412
89347999524,5cyclictest131rcu_preempt22:01:401
893479995232,12cyclictest23-21ksoftirqd/122:40:001
89347499521,47cyclictest899374-21sshd21:02:400
89347499521,47cyclictest897021-21sshd20:17:380
89348699517,11cyclictest37-21ksoftirqd/300:08:293
89348699517,11cyclictest37-21ksoftirqd/300:08:293
89348699516,11cyclictest37-21ksoftirqd/322:12:483
89348699516,11cyclictest37-21ksoftirqd/321:40:243
89348699516,10cyclictest37-21ksoftirqd/321:29:343
89348699515,8cyclictest37-21ksoftirqd/323:26:113
89348699515,7cyclictest37-21ksoftirqd/319:25:013
893486995143,6cyclictest37-21ksoftirqd/323:07:173
893486995135,12cyclictest1-21systemd00:00:013
893486995110,12cyclictest37-21ksoftirqd/322:50:553
893483995144,4cyclictest867038-21kworker/2:319:15:032
893483995144,4cyclictest867038-21kworker/2:319:15:032
893483995143,5cyclictest925658-21kworker/2:322:55:422
893483995143,5cyclictest894489-21kworker/2:121:56:552
893483995143,5cyclictest894489-21kworker/2:121:56:542
893483995143,4cyclictest894489-21kworker/2:122:25:022
893483995142,6cyclictest958508-21kworker/2:200:20:122
893483995142,5cyclictest894489-21kworker/2:122:25:592
893483995141,6cyclictest925658-21kworker/2:323:36:442
893479995134,13cyclictest275-21dbus-daemon19:15:011
89347999512,10cyclictest23-21ksoftirqd/100:00:001
89347999511,24cyclictest0-21swapper/123:47:561
89347499511,46cyclictest0-21swapper/020:07:440
89348699507,8cyclictest37-21ksoftirqd/321:30:443
89348699506,8cyclictest37-21ksoftirqd/321:55:443
89348699506,8cyclictest37-21ksoftirqd/321:55:443
89348699504,9cyclictest37-21ksoftirqd/321:17:473
89348699504,10cyclictest37-21ksoftirqd/322:31:133
89348699502,14cyclictest37-21ksoftirqd/322:16:523
89348699501,7cyclictest896534-21munin-run20:10:003
893486995010,8cyclictest37-21ksoftirqd/321:10:593
893483995043,4cyclictest894489-21kworker/2:121:33:502
893483995042,5cyclictest925658-21kworker/2:323:28:562
893483995042,5cyclictest894489-21kworker/2:121:25:062
893483995041,6cyclictest958508-21kworker/2:200:16:272
893483995041,5cyclictest925658-21kworker/2:323:15:302
893483995041,5cyclictest894489-21kworker/2:122:35:382
893483995040,7cyclictest925658-21kworker/2:322:49:392
89347999504,9cyclictest23-21ksoftirqd/100:25:511
893479995023,17cyclictest23-21ksoftirqd/100:09:411
893479995023,17cyclictest23-21ksoftirqd/100:09:411
89348699492,13cyclictest37-21ksoftirqd/300:26:093
89348699492,10cyclictest37-21ksoftirqd/322:04:223
893486994911,12cyclictest37-21ksoftirqd/321:10:023
893483994941,5cyclictest994046-21kworker/2:100:34:262
893483994941,5cyclictest958508-21kworker/2:200:02:492
893483994941,5cyclictest867038-21kworker/2:321:06:242
893483994941,4cyclictest894489-21kworker/2:122:44:432
893483994940,6cyclictest958508-21kworker/2:200:05:472
893483994940,6cyclictest958508-21kworker/2:200:05:462
893483994940,6cyclictest894489-21kworker/2:122:12:412
893483994940,5cyclictest925658-21kworker/2:322:51:272
893483994939,7cyclictest894489-21kworker/2:121:25:002
893479994941,5cyclictest23-21ksoftirqd/121:38:441
89347999493,6cyclictest23-21ksoftirqd/123:22:481
893479994913,5cyclictest131rcu_preempt21:11:001
89347999491,17cyclictest0-21swapper/123:38:401
89347999491,12cyclictest954081-21munin-run23:05:011
89348699489,9cyclictest37-21ksoftirqd/320:45:033
89348699487,6cyclictest37-21ksoftirqd/321:39:413
89348699487,10cyclictest37-21ksoftirqd/319:57:373
89348699484,11cyclictest37-21ksoftirqd/319:30:013
89348699483,8cyclictest37-21ksoftirqd/322:20:203
893483994840,5cyclictest958508-21kworker/2:200:11:332
893483994840,5cyclictest894489-21kworker/2:122:30:482
893483994840,5cyclictest894489-21kworker/2:121:47:582
893483994839,5cyclictest925658-21kworker/2:300:29:372
893479994824,9cyclictest23-21ksoftirqd/119:25:011
893479994816,28cyclictest131rcu_preempt23:31:211
893479994814,6cyclictest131rcu_preempt23:15:301
89347999481,20cyclictest0-21swapper/121:16:061
89347999481,13cyclictest23-21ksoftirqd/121:10:011
89348699478,10cyclictest37-21ksoftirqd/323:25:013
89348699476,7cyclictest37-21ksoftirqd/320:35:023
89348699476,10cyclictest37-21ksoftirqd/319:10:023
89348699476,10cyclictest37-21ksoftirqd/319:10:013
89348699472,6cyclictest37-21ksoftirqd/322:56:403
893483994739,5cyclictest958508-21kworker/2:223:50:352
893483994739,5cyclictest894489-21kworker/2:121:41:402
893483994739,5cyclictest867038-21kworker/2:321:12:312
893483994736,7cyclictest867038-21kworker/2:320:00:002
89347999476,6cyclictest131rcu_preempt22:13:051
89347999474,13cyclictest23-21ksoftirqd/122:07:471
89347999473,5cyclictest131rcu_preempt22:33:071
893479994715,14cyclictest951708-21cron23:00:001
893479994713,6cyclictest131rcu_preempt23:44:251
89347999471,22cyclictest0-21swapper/121:55:141
89347999471,22cyclictest0-21swapper/121:55:131
89347999471,22cyclictest0-21swapper/121:51:561
89347999471,20cyclictest0-21swapper/100:37:521
89348699465,12cyclictest37-21ksoftirqd/319:30:363
89348699465,12cyclictest37-21ksoftirqd/319:30:363
89348699464,8cyclictest37-21ksoftirqd/323:35:483
89348699462,8cyclictest37-21ksoftirqd/322:36:353
893486994622,17cyclictest275-21dbus-daemon19:55:013
893483994638,5cyclictest925658-21kworker/2:323:07:082
893483994638,5cyclictest867038-21kworker/2:320:40:552
893483994638,5cyclictest867038-21kworker/2:319:33:312
893483994638,5cyclictest867038-21kworker/2:319:33:302
893483994637,6cyclictest867038-21kworker/2:319:35:372
893483994611,25cyclictest291rcuc/220:10:022
89347999468,15cyclictest131rcu_preempt21:49:171
89347999467,6cyclictest131rcu_preempt00:12:161
89347999465,8cyclictest23-21ksoftirqd/100:35:011
893479994636,7cyclictest23-21ksoftirqd/122:40:111
89347999463,5cyclictest131rcu_preempt23:29:591
893479994632,9cyclictest3467602-21systemd-journal20:10:001
89347999463,11cyclictest23-21ksoftirqd/123:15:011
893479994625,8cyclictest23-21ksoftirqd/119:20:011
893479994625,8cyclictest23-21ksoftirqd/119:20:001
893479994610,5cyclictest131rcu_preempt23:52:211
89347499462,39cyclictest893472-21cat19:10:020
89347499462,39cyclictest893472-21cat19:10:010
893483994537,5cyclictest867038-21kworker/2:319:11:232
893479994532,9cyclictest896001-21cron20:00:011
893479994516,16cyclictest23-21ksoftirqd/123:05:221
893479994511,8cyclictest23-21ksoftirqd/122:26:211
89348699441,8cyclictest0-21swapper/320:17:003
893483994436,5cyclictest867038-21kworker/2:319:10:032
893483994436,5cyclictest867038-21kworker/2:319:10:022
893483994429,11cyclictest899118-21cron21:00:002
89347999445,8cyclictest23-21ksoftirqd/122:25:021
893479994434,7cyclictest23-21ksoftirqd/121:30:101
893479994428,11cyclictest23-21ksoftirqd/121:05:001
89347999441,26cyclictest221rcuc/121:30:031
893483994330,9cyclictest30-21ksoftirqd/220:15:012
89347999439,5cyclictest131rcu_preempt22:15:421
893479994324,13cyclictest275-21dbus-daemon20:35:011
89347999431,16cyclictest0-21swapper/100:22:221
89348699427,5cyclictest37-21ksoftirqd/320:50:023
89348699425,7cyclictest37-21ksoftirqd/320:13:393
893486994226,12cyclictest3467602-21systemd-journal20:39:593
893486994225,13cyclictest275-21dbus-daemon20:55:013
893486994225,13cyclictest275-21dbus-daemon20:55:003
89348699422,10cyclictest37-21ksoftirqd/319:11:483
893483994234,5cyclictest867038-21kworker/2:320:22:082
893483994227,11cyclictest30-21ksoftirqd/219:45:012
89347999425,5cyclictest131rcu_preempt19:50:011
893479994233,6cyclictest23-21ksoftirqd/120:14:381
893479994224,8cyclictest898597-21sshd20:49:381
89348699417,10cyclictest37-21ksoftirqd/319:50:033
89348699411,11cyclictest0-21swapper/320:04:593
893483994134,4cyclictest867038-21kworker/2:320:32:312
893479994131,6cyclictest274-21cron20:20:001
893479994111,21cyclictest221rcuc/119:10:031
893479994111,21cyclictest221rcuc/119:10:021
89348699406,9cyclictest37-21ksoftirqd/320:22:003
893479994031,5cyclictest309-21in:imuxsock19:39:591
893479994028,8cyclictest309-21in:imuxsock19:30:011
89348699392,11cyclictest37-21ksoftirqd/319:36:513
893483993928,7cyclictest898791-21cat20:50:012
893483993926,8cyclictest30-21ksoftirqd/220:19:592
893479993929,8cyclictest23-21ksoftirqd/120:57:401
893479993926,9cyclictest309-21in:imuxsock20:37:391
893479993914,5cyclictest131rcu_preempt20:00:361
89348399389,6cyclictest131rcu_preempt20:40:012
893483993832,3cyclictest867038-21kworker/2:319:46:062
893479993827,8cyclictest23-21ksoftirqd/119:35:011
893479993827,8cyclictest23-21ksoftirqd/119:35:001
89347999382,11cyclictest23-21ksoftirqd/119:50:041
89348699371,10cyclictest0-21swapper/320:59:393
89348699364,9cyclictest37-21ksoftirqd/320:30:023
893483993628,5cyclictest867038-21kworker/2:319:22:362
893479993626,6cyclictest897315-21cron20:24:591
89348699350,5cyclictest150-21pr/ttyS219:17:373
89348699350,5cyclictest150-21pr/ttyS219:17:363
89348399356,9cyclictest30-21ksoftirqd/220:53:372
89348399356,9cyclictest30-21ksoftirqd/220:53:372
893479993527,6cyclictest23-21ksoftirqd/120:30:021
893479993523,9cyclictest23-21ksoftirqd/119:45:011
893479993519,11cyclictest1-21systemd20:45:001
893483993423,7cyclictest897564-21cron20:30:012
893483993420,10cyclictest895851-21sort19:55:022
89347999348,6cyclictest131rcu_preempt20:53:021
89347999348,6cyclictest131rcu_preempt20:53:021
89348699311,6cyclictest0-21swapper/319:40:363
893483993022,6cyclictest30-21ksoftirqd/220:01:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional