You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 10:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Tue May 12, 2026 00:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3917809991301,3cyclictest131rcu_preempt21:50:021
3917806998137,35cyclictest12-21ksoftirqd/000:00:000
3917806997625,3cyclictest0-21swapper/022:45:020
3917806997334,18cyclictest131rcu_preempt00:34:560
3917806997334,18cyclictest131rcu_preempt00:34:550
3917813997119,12cyclictest30-21ksoftirqd/221:10:032
391781399673,11cyclictest30-21ksoftirqd/200:00:002
3917806996753,11cyclictest16950irq/224-800000023:55:000
3917806996753,11cyclictest16950irq/224-800000023:55:000
391781399658,9cyclictest30-21ksoftirqd/220:55:012
3917813996518,11cyclictest30-21ksoftirqd/223:50:032
391780699646,10cyclictest12-21ksoftirqd/023:07:350
3917806996429,8cyclictest12-21ksoftirqd/023:47:410
3917806996418,4cyclictest3989018-21rm23:24:570
391780699634,7cyclictest12-21ksoftirqd/021:11:500
391780699633,9cyclictest12-21ksoftirqd/000:15:550
3917806996320,33cyclictest16850irq/134-800000023:37:110
3917806996311,9cyclictest12-21ksoftirqd/022:00:020
3917813996217,9cyclictest30-21ksoftirqd/200:40:022
3917806996222,5cyclictest16850irq/134-800000022:40:030
3917806996222,5cyclictest16850irq/134-800000022:40:020
3917806996219,9cyclictest12-21ksoftirqd/023:30:020
3917806996213,10cyclictest12-21ksoftirqd/021:35:010
391781399616,7cyclictest30-21ksoftirqd/222:45:002
391781399616,10cyclictest30-21ksoftirqd/200:16:372
3917806996111,11cyclictest12-21ksoftirqd/021:50:020
3917813996038,17cyclictest1-21systemd21:35:012
3917813996021,8cyclictest30-21ksoftirqd/221:11:142
3917813996017,8cyclictest30-21ksoftirqd/222:38:032
3917813996017,8cyclictest30-21ksoftirqd/222:38:032
3917813996016,8cyclictest30-21ksoftirqd/221:20:012
3917813996015,12cyclictest30-21ksoftirqd/221:55:022
3917813996015,12cyclictest30-21ksoftirqd/221:55:022
3917813996015,10cyclictest30-21ksoftirqd/222:05:022
3917809996037,17cyclictest275-21dbus-daemon22:05:011
391780699603,52cyclictest3955459-21ssh22:15:030
391780699603,22cyclictest12-21ksoftirqd/000:23:250
3917806996013,9cyclictest12-21ksoftirqd/021:37:380
391781399596,7cyclictest30-21ksoftirqd/222:30:022
391781399596,7cyclictest30-21ksoftirqd/222:30:022
3917813995915,6cyclictest131rcu_preempt22:08:172
391780699597,8cyclictest12-21ksoftirqd/023:01:490
3917806995928,15cyclictest3938729-21ssh21:40:120
3917806995915,9cyclictest12-21ksoftirqd/023:11:020
391781699587,6cyclictest37-21ksoftirqd/321:40:023
391780699586,7cyclictest12-21ksoftirqd/000:25:570
391780699584,6cyclictest12-21ksoftirqd/022:00:040
3917806995821,33cyclictest16850irq/134-800000022:34:570
391780699581,12cyclictest12-21ksoftirqd/023:41:210
391781399577,9cyclictest30-21ksoftirqd/222:34:592
391781399577,7cyclictest30-21ksoftirqd/200:25:012
391781399576,10cyclictest30-21ksoftirqd/221:44:432
391781399573,8cyclictest30-21ksoftirqd/219:50:012
391781399571,8cyclictest3943454-21ssh21:50:022
391780699574,9cyclictest12-21ksoftirqd/023:20:020
3917806995720,8cyclictest131rcu_preempt22:24:210
3917806995719,33cyclictest16850irq/134-800000023:31:110
391781699565,8cyclictest37-21ksoftirqd/323:20:013
391781699565,7cyclictest131rcu_preempt00:00:003
391781399566,9cyclictest30-21ksoftirqd/223:20:032
3917813995614,5cyclictest131rcu_preempt21:40:022
3917813995614,5cyclictest131rcu_preempt00:02:522
3917813995611,5cyclictest131rcu_preempt23:40:302
3917813995610,5cyclictest131rcu_preempt23:26:092
391780699565,4cyclictest131rcu_preempt00:06:360
3917806995618,33cyclictest16850irq/134-800000022:25:340
3917806995618,33cyclictest16850irq/134-800000022:25:330
3917806995610,9cyclictest12-21ksoftirqd/000:35:040
391781699553,13cyclictest37-21ksoftirqd/322:05:023
391781399557,10cyclictest30-21ksoftirqd/220:25:022
391781399557,10cyclictest30-21ksoftirqd/220:25:012
391781399556,7cyclictest30-21ksoftirqd/200:32:092
391781399556,7cyclictest30-21ksoftirqd/200:32:092
391781399555,10cyclictest30-21ksoftirqd/221:56:402
391781399552,9cyclictest30-21ksoftirqd/223:40:012
3917813995512,11cyclictest30-21ksoftirqd/200:30:002
3917813995511,4cyclictest131rcu_preempt00:05:462
391780699555,9cyclictest121ksoftirqd/020:41:350
391780699554,11cyclictest12-21ksoftirqd/019:15:010
3917806995531,21cyclictest16850irq/134-800000021:54:160
3917806995531,21cyclictest16850irq/134-800000021:54:160
3917806995526,24cyclictest3932818-21ssh21:28:160
391780699552,38cyclictest4012633-21ssh00:13:500
3917806995518,34cyclictest16850irq/134-800000022:59:340
3917806995518,33cyclictest16850irq/134-800000000:02:240
3917806995517,34cyclictest16850irq/134-800000022:06:110
3917806995513,11cyclictest12-21ksoftirqd/021:10:030
3917806995512,10cyclictest0-21swapper/020:00:000
391781699546,9cyclictest37-21ksoftirqd/321:21:153
3917816995444,7cyclictest361rcuc/321:34:593
391781399548,10cyclictest3920868-21kworker/2:321:29:552
391781399546,11cyclictest30-21ksoftirqd/222:11:172
391781399541,8cyclictest30-21ksoftirqd/200:11:122
3917813995410,5cyclictest131rcu_preempt21:25:012
3917809995435,14cyclictest1-21systemd20:00:011
391780699544,14cyclictest12-21ksoftirqd/020:20:000
3917806995427,12cyclictest3927347-21ssh21:16:490
391780699542,11cyclictest12-21ksoftirqd/020:05:010
3917806995418,5cyclictest16850irq/134-800000022:20:000
3917806995416,30cyclictest16850irq/134-800000022:48:310
3917806995416,30cyclictest16850irq/134-800000022:48:310
3917806995416,30cyclictest16850irq/134-800000021:24:230
391781699534,10cyclictest37-21ksoftirqd/321:47:493
391781699533,5cyclictest131rcu_preempt23:10:013
3917816995317,26cyclictest37-21ksoftirqd/319:13:323
3917816995314,9cyclictest131rcu_preempt23:41:053
3917816995313,11cyclictest37-21ksoftirqd/321:13:383
391781399536,11cyclictest30-21ksoftirqd/219:45:002
3917813995312,6cyclictest131rcu_preempt23:53:332
3917813995312,6cyclictest131rcu_preempt23:53:322
3917813995311,5cyclictest131rcu_preempt23:04:162
3917806995314,31cyclictest16850irq/134-800000022:50:200
3917806995312,7cyclictest12-21ksoftirqd/020:40:030
3917806995310,8cyclictest151rcuc/019:21:350
391781399529,9cyclictest30-21ksoftirqd/220:20:022
391781399528,10cyclictest30-21ksoftirqd/219:25:002
391781399526,8cyclictest30-21ksoftirqd/222:18:552
3917813995213,5cyclictest131rcu_preempt23:32:092
3917813995210,6cyclictest131rcu_preempt23:23:282
3917813995210,4cyclictest131rcu_preempt22:52:402
3917809995235,11cyclictest275-21dbus-daemon22:55:011
391780699527,7cyclictest12-21ksoftirqd/020:50:010
3917806995210,4cyclictest131rcu_preempt19:40:010
391781699519,9cyclictest37-21ksoftirqd/300:28:533
391781699517,6cyclictest131rcu_preempt00:05:003
391781699517,19cyclictest131rcu_preempt21:27:083
3917816995128,17cyclictest1-21systemd22:25:023
3917816995123,18cyclictest37-21ksoftirqd/322:14:283
391781699511,5cyclictest131rcu_preempt22:28:273
391781699511,5cyclictest131rcu_preempt22:28:263
391781699511,5cyclictest131rcu_preempt21:10:013
391781399519,12cyclictest30-21ksoftirqd/222:46:252
391781399519,12cyclictest30-21ksoftirqd/222:46:252
3917813995118,5cyclictest131rcu_preempt19:10:022
3917813995115,10cyclictest30-21ksoftirqd/222:55:022
3917813995112,5cyclictest131rcu_preempt23:08:362
3917813995111,11cyclictest30-21ksoftirqd/220:30:012
391781699505,8cyclictest37-21ksoftirqd/322:59:423
391781699505,5cyclictest0-21swapper/322:36:353
391781699505,5cyclictest0-21swapper/322:36:353
3917816995024,9cyclictest37-21ksoftirqd/300:17:263
391781399509,6cyclictest131rcu_preempt19:35:022
391781399507,7cyclictest30-21ksoftirqd/219:17:012
391780699506,9cyclictest12-21ksoftirqd/020:10:020
391780699505,8cyclictest12-21ksoftirqd/020:55:020
391780699505,8cyclictest12-21ksoftirqd/019:20:010
391781699499,5cyclictest131rcu_preempt23:25:013
391781699499,13cyclictest37-21ksoftirqd/320:25:023
391781699499,13cyclictest37-21ksoftirqd/320:25:013
391781699497,5cyclictest131rcu_preempt00:10:003
3917816994921,8cyclictest131rcu_preempt21:15:323
391781399497,9cyclictest30-21ksoftirqd/222:20:492
391781399496,7cyclictest30-21ksoftirqd/221:05:022
391781399495,10cyclictest30-21ksoftirqd/220:10:002
3917813994941,5cyclictest3910684-21kworker/2:019:57:182
391781399494,13cyclictest30-21ksoftirqd/223:10:272
391780699497,7cyclictest12-21ksoftirqd/020:15:030
3917806994917,9cyclictest12-21ksoftirqd/020:35:010
391781699487,6cyclictest131rcu_preempt23:13:033
391781699487,11cyclictest37-21ksoftirqd/322:18:543
391781699482,6cyclictest131rcu_preempt20:55:013
3917816994822,17cyclictest37-21ksoftirqd/323:27:523
3917816994819,8cyclictest37-21ksoftirqd/300:12:223
391781399489,5cyclictest131rcu_preempt19:13:352
391781399487,7cyclictest30-21ksoftirqd/220:15:022
3917813994840,5cyclictest3921911-21kworker/2:020:55:552
3917813994840,5cyclictest3921911-21kworker/2:020:38:532
3917813994840,5cyclictest3911205-21kworker/2:220:03:342
3917813994840,5cyclictest3910684-21kworker/2:019:51:372
3917813994839,6cyclictest3922459-21kworker/2:120:44:002
3917809994837,6cyclictest309-21in:imuxsock00:30:001
391780999481,3cyclictest131rcu_preempt21:44:201
391780699481,12cyclictest12-21ksoftirqd/019:55:020
391781699476,8cyclictest37-21ksoftirqd/323:52:093
391781699476,8cyclictest37-21ksoftirqd/323:52:093
3917816994736,6cyclictest37-21ksoftirqd/300:31:353
3917816994736,6cyclictest37-21ksoftirqd/300:31:353
391781699473,5cyclictest131rcu_preempt20:40:003
391781699472,6cyclictest131rcu_preempt22:40:393
391781699471,6cyclictest131rcu_preempt23:40:013
3917813994739,5cyclictest3921911-21kworker/2:020:47:102
3917813994739,5cyclictest3920868-21kworker/2:320:31:012
3917813994739,5cyclictest3911205-21kworker/2:219:35:312
3917813994739,5cyclictest3910684-21kworker/2:019:28:492
391780999477,37cyclictest3969809-21ssh22:45:011
3917809994735,7cyclictest3996300-21cron23:40:001
3917809994730,12cyclictest0-21swapper/100:00:001
3917809994730,11cyclictest3467602-21systemd-journal22:26:361
3917809994730,11cyclictest3467602-21systemd-journal22:26:351
3917809994726,14cyclictest3467602-21systemd-journal19:20:011
391780699478,5cyclictest131rcu_preempt19:50:010
391781699468,5cyclictest131rcu_preempt21:56:393
391781699467,6cyclictest131rcu_preempt22:53:393
391781699465,6cyclictest131rcu_preempt20:45:013
391781699463,6cyclictest0-21swapper/323:30:173
3917816994627,8cyclictest37-21ksoftirqd/321:40:063
3917816994621,6cyclictest361rcuc/322:31:273
391780999463,12cyclictest364-21snmpd00:22:571
3917809994631,11cyclictest3960119-21systemd-run22:25:011
3917809994630,11cyclictest3467602-21systemd-journal19:50:001
391780699466,6cyclictest12-21ksoftirqd/020:25:010
391780699466,6cyclictest12-21ksoftirqd/020:25:010
391780699464,12cyclictest12-21ksoftirqd/020:30:020
391781699455,6cyclictest37-21ksoftirqd/322:45:123
391781699455,6cyclictest37-21ksoftirqd/322:45:113
3917816994534,7cyclictest37-21ksoftirqd/320:10:013
3917816994531,10cyclictest3921373-21cron20:20:003
3917816994529,12cyclictest37-21ksoftirqd/319:40:003
391781699452,6cyclictest0-21swapper/322:06:333
3917816994520,20cyclictest37-21ksoftirqd/300:38:003
3917816994520,17cyclictest37-21ksoftirqd/321:54:393
3917816994520,17cyclictest37-21ksoftirqd/321:54:383
3917816994518,21cyclictest131rcu_preempt00:22:333
3917816994516,18cyclictest131rcu_preempt23:50:033
3917816994512,6cyclictest131rcu_preempt19:19:343
3917809994533,10cyclictest23-21ksoftirqd/123:04:561
3917809994532,10cyclictest23-21ksoftirqd/121:35:101
3917809994528,11cyclictest275-21dbus-daemon23:35:011
391780699452,11cyclictest12-21ksoftirqd/019:30:020
3917806994511,4cyclictest131rcu_preempt21:01:380
391781699446,7cyclictest131rcu_preempt20:35:023
391781699442,6cyclictest131rcu_preempt20:27:353
391781699441,5cyclictest131rcu_preempt20:00:013
391781699441,4cyclictest131rcu_preempt23:00:593
391780999443,3cyclictest131rcu_preempt22:14:001
3917809994431,9cyclictest309-21in:imuxsock00:40:001
3917809994430,11cyclictest3467602-21systemd-journal21:11:361
3917809994425,14cyclictest275-21dbus-daemon00:15:011
3917809994425,13cyclictest275-21dbus-daemon21:35:011
3917809994423,15cyclictest275-21dbus-daemon23:25:011
391780699443,8cyclictest12-21ksoftirqd/019:45:000
391780699442,11cyclictest12-21ksoftirqd/019:30:020
391780699441,7cyclictest0-21swapper/019:10:020
3917806994410,5cyclictest131rcu_preempt21:00:010
3917816994333,7cyclictest37-21ksoftirqd/319:34:593
391781699431,7cyclictest3921160-21latency_hist20:15:023
3917809994334,5cyclictest3921374-21sendmail20:19:591
3917809994330,9cyclictest3964235-21sshd22:33:361
3917809994330,10cyclictest23-21ksoftirqd/123:42:561
391781699429,6cyclictest131rcu_preempt20:50:013
391781699426,6cyclictest131rcu_preempt19:55:023
391781699422,7cyclictest37-21ksoftirqd/319:50:013
3917816994214,5cyclictest131rcu_preempt19:30:023
3917809994231,8cyclictest4003662-21ssh23:55:021
3917809994231,8cyclictest4003662-21ssh23:55:021
391780999422,3cyclictest131rcu_preempt21:24:421
391780999421,3cyclictest131rcu_preempt00:35:021
391780999421,3cyclictest131rcu_preempt00:35:021
3917809994210,6cyclictest23-21ksoftirqd/119:25:011
391781699415,10cyclictest37-21ksoftirqd/321:05:013
391781699411,18cyclictest0-21swapper/320:05:013
3917809994129,7cyclictest23-21ksoftirqd/122:38:031
3917809994129,7cyclictest23-21ksoftirqd/122:38:021
3917809994128,9cyclictest309-21in:imuxsock23:00:011
3917809994127,9cyclictest23-21ksoftirqd/120:55:021
3917809994126,11cyclictest23-21ksoftirqd/100:07:411
3917809994124,7cyclictest23-21ksoftirqd/100:05:001
3917809994124,13cyclictest275-21dbus-daemon21:30:021
391780999411,3cyclictest131rcu_preempt23:16:361
391780999411,3cyclictest131rcu_preempt22:18:401
3917816994031,7cyclictest37-21ksoftirqd/319:45:013
3917816994016,6cyclictest131rcu_preempt20:55:353
3917816994011,6cyclictest131rcu_preempt19:25:033
3917809994026,11cyclictest3926622-21sh21:15:081
3917809994026,11cyclictest23-21ksoftirqd/120:40:001
3917809994025,12cyclictest1-21systemd21:55:011
3917809994025,12cyclictest1-21systemd21:55:001
391780999402,3cyclictest131rcu_preempt22:06:091
391780999401,10cyclictest3989358-21ssh23:25:111
3917809993929,7cyclictest3999196-21sshd23:45:341
3917809993927,8cyclictest23-21ksoftirqd/100:20:031
3917809993915,3cyclictest23-21ksoftirqd/121:00:001
391780999391,3cyclictest131rcu_preempt19:45:041
3917809993825,10cyclictest48-21kcompactd021:59:071
3917809993825,10cyclictest3467602-21systemd-journal20:29:591
3917809993817,6cyclictest131rcu_preempt19:15:011
391780999381,5cyclictest131rcu_preempt23:12:061
391780999376,5cyclictest131rcu_preempt19:30:011
3917809993726,8cyclictest23-21ksoftirqd/120:10:011
3917809993716,9cyclictest23-21ksoftirqd/121:05:011
3917809993627,7cyclictest23-21ksoftirqd/123:08:521
391780999362,4cyclictest131rcu_preempt22:45:261
391780999362,4cyclictest131rcu_preempt22:45:261
3917809993621,11cyclictest23-21ksoftirqd/121:09:001
3917809993620,11cyclictest275-21dbus-daemon20:50:011
3917809993616,16cyclictest275-21dbus-daemon19:55:011
391780999361,5cyclictest131rcu_preempt20:35:021
391781699351,4cyclictest131rcu_preempt19:10:013
391780999352,3cyclictest131rcu_preempt19:10:011
391780999344,5cyclictest131rcu_preempt20:25:011
391780999344,5cyclictest131rcu_preempt20:25:011
3917809993427,5cyclictest23-21ksoftirqd/120:45:011
3917809993422,9cyclictest23-21ksoftirqd/120:15:021
3917809993420,10cyclictest3919305-21systemctl19:39:591
3917809993217,12cyclictest3920618-21systemd-run20:05:001
391780999311,3cyclictest131rcu_preempt19:35:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional