You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 09:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Sun Apr 19, 2026 00:45:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8410719910598,4cyclictest858195-21sshd21:35:012
84107199912,4cyclictest131rcu_preempt20:30:022
84107199844,74cyclictest925068-21ssh00:10:012
84107699762,57cyclictest0-21swapper/323:05:013
841064997633,10cyclictest12-21ksoftirqd/021:17:170
841071997131,12cyclictest843567-21latency_hist19:55:032
841064997012,39cyclictest0-21swapper/019:56:030
841064996746,16cyclictest16950irq/224-800000023:20:010
841064996730,33cyclictest12-21ksoftirqd/022:55:490
841064996638,24cyclictest16850irq/134-800000021:13:530
841064996638,24cyclictest16850irq/134-800000021:13:520
841064996513,9cyclictest12-21ksoftirqd/000:10:020
841064996422,9cyclictest12-21ksoftirqd/022:26:560
84106899632,46cyclictest860427-21sh21:40:021
84106899632,46cyclictest860427-21sh21:40:021
841064996345,6cyclictest12-21ksoftirqd/021:26:030
841064996323,7cyclictest918910-21ssh23:55:200
841071996228,12cyclictest1-21systemd19:20:012
84106499624,9cyclictest12-21ksoftirqd/022:22:250
84106499623,11cyclictest121ksoftirqd/021:55:030
84106499613,9cyclictest12-21ksoftirqd/022:45:010
84106499613,50cyclictest899101-21latency_hist23:10:020
84106499613,50cyclictest899101-21latency_hist23:10:020
841064996133,11cyclictest908867-21ssh23:32:290
841064996133,11cyclictest908867-21ssh23:32:290
84106499607,13cyclictest12-21ksoftirqd/022:31:030
841064996020,33cyclictest12-21ksoftirqd/000:40:010
841064996012,13cyclictest12-21ksoftirqd/021:37:160
841076995931,10cyclictest37-21ksoftirqd/321:35:013
841064995926,16cyclictest873648-21ssh22:10:360
841064995913,11cyclictest12-21ksoftirqd/021:43:570
841064995913,11cyclictest12-21ksoftirqd/021:43:570
841064995826,25cyclictest853727-21munin-run21:25:000
841064995826,25cyclictest853727-21munin-run21:24:590
841076995738,15cyclictest903414-21tr23:20:003
841064995726,15cyclictest910685-21ssh23:36:310
841064995725,28cyclictest12-21ksoftirqd/021:31:380
84106499568,8cyclictest12-21ksoftirqd/023:51:030
84106499568,8cyclictest12-21ksoftirqd/023:51:030
841064995618,14cyclictest12-21ksoftirqd/023:15:000
841064995617,35cyclictest12-21ksoftirqd/000:25:020
841064995617,35cyclictest12-21ksoftirqd/000:25:020
841064995611,40cyclictest927208-21ssh00:15:020
841064995611,40cyclictest927208-21ssh00:15:020
841064995540,10cyclictest3467602-21systemd-journal21:59:590
841064995534,14cyclictest1-21systemd20:15:010
841064995528,23cyclictest885990-21ssh22:39:420
841064995520,11cyclictest12-21ksoftirqd/023:04:100
841071995445,6cyclictest916447-21kworker/2:100:26:452
84106899544,7cyclictest23-21ksoftirqd/121:35:021
84106499546,8cyclictest12-21ksoftirqd/020:10:020
84106499546,10cyclictest12-21ksoftirqd/020:55:030
841064995414,11cyclictest12-21ksoftirqd/022:10:030
841064995411,9cyclictest12-21ksoftirqd/020:25:020
841064995411,9cyclictest12-21ksoftirqd/020:25:010
84106499538,11cyclictest12-21ksoftirqd/022:54:370
84106499534,8cyclictest12-21ksoftirqd/000:00:060
84106499534,8cyclictest12-21ksoftirqd/000:00:060
841064995342,7cyclictest16950irq/224-800000020:20:000
841064995342,7cyclictest16950irq/224-800000020:20:000
84106499534,14cyclictest12-21ksoftirqd/022:05:030
841064995326,12cyclictest913181-21ssh23:42:190
841064995326,12cyclictest904141-21ssh23:21:230
841064995326,12cyclictest904141-21ssh23:21:230
84106499531,37cyclictest935728-21ssh00:34:440
84107699522,5cyclictest131rcu_preempt00:40:013
841071995244,5cyclictest871445-21kworker/2:022:25:162
841071995244,5cyclictest845488-21kworker/2:321:27:152
841071995242,5cyclictest877592-21kworker/2:223:12:302
841071995236,13cyclictest3467602-21systemd-journal23:57:022
84106899525,7cyclictest131rcu_preempt00:00:021
841068995227,18cyclictest0-21swapper/120:40:011
84106499526,8cyclictest12-21ksoftirqd/021:50:000
84106499525,6cyclictest12-21ksoftirqd/000:20:010
84106499525,10cyclictest12-21ksoftirqd/023:46:030
841064995242,6cyclictest16950irq/224-800000019:20:000
84106499522,11cyclictest12-21ksoftirqd/020:40:010
841064995217,11cyclictest12-21ksoftirqd/023:30:010
84107699515,28cyclictest871239-21ssh22:05:033
841076995127,8cyclictest37-21ksoftirqd/322:55:003
84107699511,9cyclictest843209-21latency_hist19:50:023
841071995143,5cyclictest877592-21kworker/2:223:21:372
841071995143,5cyclictest877592-21kworker/2:223:21:372
841071995143,5cyclictest871445-21kworker/2:023:19:372
841071995142,5cyclictest877592-21kworker/2:223:33:052
841071995142,5cyclictest877592-21kworker/2:223:33:052
841071995141,6cyclictest845488-21kworker/2:321:22:312
841071995141,6cyclictest845488-21kworker/2:321:22:302
841071995141,6cyclictest845488-21kworker/2:321:15:292
841071995110,5cyclictest131rcu_preempt21:00:012
841068995135,12cyclictest1-21systemd00:20:011
84106499517,9cyclictest12-21ksoftirqd/022:16:540
84106499517,11cyclictest12-21ksoftirqd/000:27:550
84106499512,12cyclictest12-21ksoftirqd/022:47:230
841071995042,5cyclictest858071-21kworker/2:121:41:272
841071995042,5cyclictest858071-21kworker/2:121:41:272
841071995041,6cyclictest877592-21kworker/2:222:49:002
841071995041,5cyclictest871445-21kworker/2:000:32:492
84106899507,6cyclictest131rcu_preempt19:50:011
841068995036,10cyclictest275-21dbus-daemon23:10:011
841068995036,10cyclictest275-21dbus-daemon23:10:011
841068995031,14cyclictest275-21dbus-daemon22:10:011
841068995018,7cyclictest131rcu_preempt19:15:021
841068995018,7cyclictest131rcu_preempt19:15:021
84106499508,9cyclictest12-21ksoftirqd/019:30:000
84106499507,8cyclictest12-21ksoftirqd/020:50:030
84106499504,35cyclictest843378-21latency_hist19:55:010
841064995032,13cyclictest275-21dbus-daemon19:25:010
84106499502,10cyclictest12-21ksoftirqd/021:05:030
84106499501,7cyclictest0-21swapper/019:40:000
84107699494,6cyclictest131rcu_preempt00:00:013
841076994912,6cyclictest131rcu_preempt22:45:003
841071994942,5cyclictest846267-21kworker/2:221:35:352
841071994941,5cyclictest877592-21kworker/2:223:36:382
841071994941,5cyclictest871445-21kworker/2:023:40:442
841071994941,5cyclictest858071-21kworker/2:122:05:182
841071994940,6cyclictest916447-21kworker/2:100:38:562
841071994940,6cyclictest871445-21kworker/2:022:10:402
841071994940,5cyclictest858071-21kworker/2:121:47:512
841071994939,6cyclictest842011-21kworker/2:120:25:022
841071994939,6cyclictest842011-21kworker/2:120:25:022
841071994939,6cyclictest842011-21kworker/2:120:19:342
841071994939,6cyclictest842011-21kworker/2:120:19:332
841071994938,7cyclictest871445-21kworker/2:022:53:502
841076994829,15cyclictest275-21dbus-daemon21:30:003
84107699482,4cyclictest131rcu_preempt23:55:023
84107699482,4cyclictest131rcu_preempt23:55:013
84107699482,3cyclictest131rcu_preempt23:27:083
841076994821,9cyclictest0-21swapper/300:31:493
841076994810,3cyclictest131rcu_preempt00:03:223
841076994810,3cyclictest131rcu_preempt00:03:213
841071994841,4cyclictest858071-21kworker/2:121:52:052
841071994840,5cyclictest846267-21kworker/2:221:56:132
841071994840,5cyclictest845488-21kworker/2:321:14:332
841071994840,5cyclictest845488-21kworker/2:321:14:332
841071994838,6cyclictest916447-21kworker/2:100:19:222
841071994837,7cyclictest877592-21kworker/2:222:31:342
841071994832,12cyclictest275-21dbus-daemon20:00:012
841068994831,13cyclictest275-21dbus-daemon21:50:001
84106499488,8cyclictest131rcu_preempt19:10:020
84106499487,12cyclictest12-21ksoftirqd/020:30:010
84107699475,3cyclictest131rcu_preempt21:14:573
84107699475,3cyclictest131rcu_preempt21:14:573
84107699475,3cyclictest131rcu_preempt00:08:283
84107699473,4cyclictest131rcu_preempt22:16:253
841076994731,12cyclictest275-21dbus-daemon22:15:003
841071994739,5cyclictest877592-21kworker/2:223:08:432
841071994739,5cyclictest877592-21kworker/2:223:08:432
841071994739,5cyclictest877592-21kworker/2:223:02:542
841071994739,5cyclictest877592-21kworker/2:222:41:182
841071994739,5cyclictest871445-21kworker/2:022:21:312
84106899475,6cyclictest131rcu_preempt19:25:011
84106899473,3cyclictest131rcu_preempt22:35:011
84106899471,3cyclictest131rcu_preempt22:51:201
84107699465,4cyclictest131rcu_preempt21:47:193
84107699464,3cyclictest131rcu_preempt22:20:203
84107699463,3cyclictest131rcu_preempt23:07:013
84107699463,3cyclictest131rcu_preempt23:07:003
84107699463,3cyclictest131rcu_preempt21:51:593
84107699462,5cyclictest131rcu_preempt22:40:033
84107699462,4cyclictest131rcu_preempt20:40:023
84107199464,23cyclictest291rcuc/219:20:012
841071994638,5cyclictest877592-21kworker/2:222:39:592
841071994638,5cyclictest871445-21kworker/2:022:18:302
841071994638,5cyclictest842011-21kworker/2:120:30:252
841071994632,10cyclictest30-21ksoftirqd/200:12:332
841071994632,10cyclictest30-21ksoftirqd/200:12:332
841071994631,10cyclictest275-21dbus-daemon21:10:012
84106899464,4cyclictest131rcu_preempt00:12:001
84106899464,4cyclictest131rcu_preempt00:12:001
84106899464,3cyclictest131rcu_preempt22:20:121
841068994634,9cyclictest23-21ksoftirqd/121:10:001
84106899463,3cyclictest131rcu_preempt22:11:371
84106899461,5cyclictest131rcu_preempt00:00:051
84106899461,5cyclictest131rcu_preempt00:00:051
84106899461,3cyclictest131rcu_preempt22:01:131
84106499467,8cyclictest12-21ksoftirqd/020:35:020
84106499463,9cyclictest846779-21modprobe21:00:000
84106499461,7cyclictest0-21swapper/019:45:030
84106499461,5cyclictest0-21swapper/019:15:010
84106499461,5cyclictest0-21swapper/019:15:000
84107699455,3cyclictest131rcu_preempt23:20:033
84107699455,3cyclictest131rcu_preempt23:20:033
84107699454,4cyclictest131rcu_preempt23:33:033
84107699454,4cyclictest131rcu_preempt23:33:033
84107699454,3cyclictest131rcu_preempt23:10:053
84107699453,4cyclictest131rcu_preempt00:20:363
84107699453,4cyclictest131rcu_preempt00:20:363
841076994532,10cyclictest1-21systemd23:50:003
841076994530,11cyclictest1-21systemd20:00:013
841076994526,16cyclictest37-21ksoftirqd/319:40:013
84107699452,4cyclictest131rcu_preempt22:56:023
84107699452,4cyclictest131rcu_preempt00:17:193
841076994522,6cyclictest37-21ksoftirqd/322:05:423
841076994511,6cyclictest131rcu_preempt22:34:453
84106899454,3cyclictest131rcu_preempt22:48:401
84106899454,3cyclictest131rcu_preempt21:21:171
84106899454,3cyclictest131rcu_preempt21:21:171
84106899453,4cyclictest131rcu_preempt23:17:581
841068994533,9cyclictest915623-21sshd23:48:021
84106899452,4cyclictest131rcu_preempt00:31:081
84106899451,5cyclictest131rcu_preempt22:45:011
84106899451,3cyclictest131rcu_preempt21:36:511
841064994535,6cyclictest12-21ksoftirqd/019:35:010
84106499451,12cyclictest12-21ksoftirqd/021:05:020
84107699444,3cyclictest131rcu_preempt22:46:063
84107699444,3cyclictest131rcu_preempt21:43:243
84107699444,3cyclictest131rcu_preempt21:43:243
84107699444,3cyclictest131rcu_preempt20:45:023
84107699443,4cyclictest131rcu_preempt00:29:573
84107699443,4cyclictest131rcu_preempt00:13:083
84107699443,4cyclictest131rcu_preempt00:13:083
84107699443,3cyclictest131rcu_preempt23:42:013
84107699443,3cyclictest131rcu_preempt23:38:433
84107699443,3cyclictest131rcu_preempt21:20:383
84107699443,3cyclictest131rcu_preempt21:20:383
84107699442,3cyclictest131rcu_preempt22:25:103
84107699442,3cyclictest131rcu_preempt21:56:373
84107699442,3cyclictest131rcu_preempt21:05:003
841076994421,7cyclictest37-21ksoftirqd/320:50:013
84107699441,5cyclictest131rcu_preempt19:55:003
84107199444,4cyclictest131rcu_preempt19:10:032
84107199444,4cyclictest131rcu_preempt19:10:022
841071994436,5cyclictest871445-21kworker/2:022:56:302
841071994436,5cyclictest858071-21kworker/2:122:03:022
841071994429,11cyclictest275-21dbus-daemon00:05:012
841071994429,11cyclictest275-21dbus-daemon00:05:012
84106899445,3cyclictest131rcu_preempt23:14:111
84106899444,3cyclictest131rcu_preempt00:29:171
84106899443,3cyclictest131rcu_preempt23:35:001
84106899443,3cyclictest131rcu_preempt23:35:001
84106899443,3cyclictest131rcu_preempt21:55:051
84106899443,3cyclictest131rcu_preempt20:45:001
841068994430,10cyclictest1-21systemd00:25:011
841068994430,10cyclictest1-21systemd00:25:011
84106899442,5cyclictest131rcu_preempt23:40:021
841068994425,13cyclictest1-21systemd20:00:001
84106899442,4cyclictest131rcu_preempt21:50:141
84106899442,3cyclictest131rcu_preempt23:42:321
84106899442,3cyclictest131rcu_preempt21:12:201
84106899442,3cyclictest131rcu_preempt21:12:201
84106899441,5cyclictest131rcu_preempt21:05:011
84106899441,4cyclictest131rcu_preempt23:23:531
84106899441,4cyclictest131rcu_preempt23:23:531
84106899441,3cyclictest131rcu_preempt23:51:371
84106899441,3cyclictest131rcu_preempt23:51:371
84106899441,3cyclictest131rcu_preempt21:27:071
84106899441,3cyclictest131rcu_preempt00:09:541
841064994431,8cyclictest3467602-21systemd-journal20:40:030
84107699434,3cyclictest131rcu_preempt21:16:553
84107699433,3cyclictest131rcu_preempt21:36:363
841071994337,3cyclictest846267-21kworker/2:221:05:002
84106899437,7cyclictest23-21ksoftirqd/123:01:011
84106899433,3cyclictest131rcu_preempt22:17:201
84106899433,3cyclictest131rcu_preempt21:15:371
84106899433,3cyclictest131rcu_preempt21:00:001
84106899433,3cyclictest131rcu_preempt19:20:001
841068994327,12cyclictest275-21dbus-daemon00:40:011
84106899431,5cyclictest131rcu_preempt22:30:011
84106899431,4cyclictest131rcu_preempt22:58:161
84106899431,4cyclictest131rcu_preempt22:36:581
84106899431,10cyclictest842826-21cron19:45:011
84106499437,7cyclictest12-21ksoftirqd/019:50:020
841064994330,10cyclictest12-21ksoftirqd/020:05:010
841076994231,8cyclictest37-21ksoftirqd/320:54:023
84107699421,7cyclictest844170-21systemd-run20:10:003
841071994236,3cyclictest871445-21kworker/2:023:28:492
841071994235,4cyclictest916447-21kworker/2:100:20:152
841071994235,4cyclictest916447-21kworker/2:100:20:142
841068994230,9cyclictest845185-21sshd20:28:011
84106899422,3cyclictest131rcu_preempt20:50:021
84106899421,4cyclictest131rcu_preempt20:10:021
84106899421,3cyclictest131rcu_preempt23:25:191
841076994127,11cyclictest37-21ksoftirqd/319:30:003
84107699411,20cyclictest0-21swapper/320:29:593
841071994133,5cyclictest842011-21kworker/2:120:41:412
841071994132,5cyclictest871445-21kworker/2:023:47:032
841071994130,7cyclictest842558-21modprobe19:39:592
841071994111,6cyclictest131rcu_preempt19:45:182
841068994124,13cyclictest275-21dbus-daemon19:40:001
84106899411,3cyclictest131rcu_preempt19:30:031
84107699404,5cyclictest131rcu_preempt19:15:013
84107699404,5cyclictest131rcu_preempt19:15:003
84107699404,3cyclictest131rcu_preempt20:35:023
84107699403,3cyclictest131rcu_preempt20:15:033
84107699403,3cyclictest131rcu_preempt20:15:023
84107699403,3cyclictest131rcu_preempt19:20:023
84107699401,3cyclictest131rcu_preempt20:25:023
84107699401,3cyclictest131rcu_preempt20:25:023
841071994032,6cyclictest30-21ksoftirqd/220:15:012
841071994030,6cyclictest871445-21kworker/2:023:52:102
841071994030,6cyclictest871445-21kworker/2:023:52:102
84106899407,5cyclictest131rcu_preempt20:20:011
84106899407,5cyclictest131rcu_preempt20:20:001
84106899404,3cyclictest131rcu_preempt19:35:001
84106899402,3cyclictest131rcu_preempt19:10:011
84106899402,33cyclictest23-21ksoftirqd/120:55:031
84106899401,3cyclictest131rcu_preempt20:35:021
841076993918,5cyclictest37-21ksoftirqd/321:10:003
84107699391,4cyclictest131rcu_preempt19:43:013
84107199394,5cyclictest131rcu_preempt19:10:022
841071993929,5cyclictest309-21in:imuxsock20:05:002
84106899392,3cyclictest131rcu_preempt20:25:021
84106899392,3cyclictest131rcu_preempt20:25:021
84107699384,3cyclictest131rcu_preempt20:05:023
84107699383,5cyclictest131rcu_preempt19:20:003
841068993828,7cyclictest844051-21cat20:05:021
841076993718,13cyclictest37-21ksoftirqd/319:35:003
841076993710,7cyclictest131rcu_preempt20:13:023
841071993728,6cyclictest841263-21kworker/2:019:27:162
841071993727,7cyclictest30-21ksoftirqd/220:10:032
841068993727,7cyclictest843357-21sshd19:52:401
84107699363,5cyclictest131rcu_preempt21:00:013
841071993627,5cyclictest842011-21kworker/2:119:30:142
841071993624,9cyclictest30-21ksoftirqd/219:45:012
84106899352,3cyclictest131rcu_preempt20:15:021
841071993426,5cyclictest845714-21latency_hist20:40:022
84107699331,5cyclictest0-21swapper/319:10:013
841071993225,5cyclictest846564-21latency_hist20:55:022
841071993120,8cyclictest846215-21sshd20:47:492
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional