You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 12:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Sep 22, 2026 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
255532170105,33sleep10-21swapper/119:07:201
254932164106,28sleep30-21swapper/319:06:353
255742161108,30sleep00-21swapper/019:07:360
257352160104,27sleep20-21swapper/219:09:422
131482690,10sleep012-21ksoftirqd/019:55:090
18672630,1sleep20-21swapper/200:36:472
85732620,1sleep20-21swapper/223:30:172
253012600,2sleep20-21swapper/200:30:202
189232600,9sleep122-21ksoftirqd/123:37:431
296332590,2sleep00-21swapper/021:21:160
296332590,2sleep00-21swapper/021:21:160
274012560,2sleep30-21swapper/323:44:263
327652550,2sleep00-21swapper/021:47:570
234902520,3chrt0-21swapper/100:29:381
279002510,1sleep00-21swapper/000:32:060
244252510,3sleep1211rcuc/122:05:251
4082500,2chrt0-21swapper/321:48:043
165192480,1sleep10-21swapper/122:00:141
325112430,2sleep10-21swapper/123:47:501
208272420,1sleep020823-21sh22:50:530
133122420,3sleep213314-21grep19:55:122
268302340,2chrt0-21swapper/121:43:461
148882340,2chrt0-21swapper/323:59:113
192822330,1sleep00-21swapper/000:01:480
2623499309,6cyclictest22-21ksoftirqd/100:02:291
26234993017,8cyclictest24336-21ssh00:30:041
2623499301,26cyclictest3333-21/usr/sbin/munin19:30:201
107522300,2sleep012-21ksoftirqd/023:55:230
26236992917,8cyclictest52950irq/141-enp3s0-21:15:123
2623499291,4cyclictest140762sleep119:55:191
26233992910,7cyclictest560-21dbus-daemon23:05:000
2623599281,25cyclictest12623-21sshd19:51:282
2623499281,25cyclictest0-21swapper/120:55:341
2623499281,25cyclictest0-21swapper/120:18:081
2623399281,25cyclictest0-21swapper/020:12:250
199802280,1sleep20-21swapper/200:02:312
118662280,2sleep00-21swapper/022:20:200
68992270,2sleep00-21swapper/023:52:530
293552270,2chrt0-21swapper/019:15:220
2623699271,24cyclictest0-21swapper/320:43:273
2623699271,24cyclictest0-21swapper/320:06:083
2623699271,24cyclictest0-21swapper/319:53:463
26235992722,3cyclictest1689-21ls19:25:262
2623599271,2cyclictest0-21swapper/220:18:472
2623599271,24cyclictest0-21swapper/220:07:112
2623599270,25cyclictest3378-21ntpq19:30:222
2623499273,12cyclictest22-21ksoftirqd/122:24:501
2623499272,16cyclictest22-21ksoftirqd/123:33:001
2623499271,25cyclictest0-21swapper/119:51:341
2623499271,24cyclictest0-21swapper/121:01:111
2623499271,24cyclictest0-21swapper/120:50:221
2623499271,24cyclictest0-21swapper/120:49:171
2623499271,24cyclictest0-21swapper/120:40:061
2623499271,24cyclictest0-21swapper/120:36:221
2623499271,24cyclictest0-21swapper/120:30:421
2623499271,24cyclictest0-21swapper/120:11:241
2623499271,24cyclictest0-21swapper/120:06:411
2623499271,24cyclictest0-21swapper/120:00:591
2623499271,24cyclictest0-21swapper/119:46:291
2623499271,24cyclictest0-21swapper/119:35:521
2623499271,24cyclictest0-21swapper/119:28:261
2623499271,24cyclictest0-21swapper/119:20:241
26234992711,3cyclictest141rcu_preempt00:07:181
26233992725,1cyclictest0-21swapper/019:54:240
26233992724,2cyclictest0-21swapper/020:33:230
26233992724,2cyclictest0-21swapper/020:01:180
26233992724,2cyclictest0-21swapper/019:37:320
2623399271,25cyclictest0-21swapper/020:58:480
2623399271,25cyclictest0-21swapper/020:29:360
2623399271,25cyclictest0-21swapper/020:05:590
2623399271,25cyclictest0-21swapper/019:30:320
2623399271,24cyclictest0-21swapper/021:00:040
2623399271,24cyclictest0-21swapper/020:50:080
2623399271,24cyclictest0-21swapper/020:41:030
2623399271,24cyclictest0-21swapper/020:37:060
2623399271,24cyclictest0-21swapper/020:15:090
2623399271,24cyclictest0-21swapper/019:46:330
2623399271,24cyclictest0-21swapper/019:41:590
2623399271,24cyclictest0-21swapper/019:41:580
2623699261,23cyclictest0-21swapper/321:01:203
2623699261,23cyclictest0-21swapper/320:55:453
2623699261,23cyclictest0-21swapper/320:54:583
2623699261,23cyclictest0-21swapper/320:35:453
2623699261,23cyclictest0-21swapper/320:17:443
2623699261,23cyclictest0-21swapper/320:12:213
2623699261,23cyclictest0-21swapper/319:57:443
2623699261,23cyclictest0-21swapper/319:43:533
2623699261,23cyclictest0-21swapper/319:43:533
2623699261,23cyclictest0-21swapper/319:35:133
2623699261,23cyclictest0-21swapper/319:33:333
2623699261,23cyclictest0-21swapper/319:22:093
26235992623,2cyclictest0-21swapper/220:37:212
2623599261,23cyclictest0-21swapper/220:50:342
2623599261,23cyclictest0-21swapper/220:47:592
2623599261,23cyclictest0-21swapper/220:40:532
2623599261,23cyclictest0-21swapper/220:30:372
2623599261,23cyclictest0-21swapper/220:28:532
2623599261,23cyclictest0-21swapper/219:46:132
2623599261,23cyclictest0-21swapper/219:41:402
2623599261,23cyclictest0-21swapper/219:41:392
2623499267,5cyclictest22-21ksoftirqd/100:24:221
26234992624,1cyclictest0-21swapper/120:23:021
2623499260,24cyclictest0-21swapper/119:40:031
2623499260,24cyclictest0-21swapper/119:40:021
26233992624,1cyclictest0-21swapper/020:46:040
26233992624,1cyclictest0-21swapper/020:23:030
2623399261,24cyclictest0-21swapper/019:28:400
2623399261,1cyclictest0-21swapper/019:20:230
161132260,2sleep20-21swapper/200:23:592
26236992523,1cyclictest0-21swapper/320:30:023
26236992522,2cyclictest0-21swapper/320:25:343
26236992522,2cyclictest0-21swapper/319:45:543
2623699251,23cyclictest0-21swapper/320:23:583
2623699250,23cyclictest0-21swapper/320:46:153
2623699250,23cyclictest0-21swapper/320:00:093
26235992523,1cyclictest0-21swapper/220:55:022
26235992523,1cyclictest0-21swapper/220:10:092
2623599251,2cyclictest0-21swapper/220:24:132
2623599251,23cyclictest0-21swapper/221:00:412
2623599251,23cyclictest0-21swapper/220:01:412
2623599251,23cyclictest0-21swapper/219:35:292
2623599251,23cyclictest0-21swapper/219:23:062
26235992510,11cyclictest26205-21packagekitd19:10:012
2623499258,15cyclictest22-21ksoftirqd/122:54:321
26234992523,1cyclictest0-21swapper/121:05:491
26234992523,1cyclictest0-21swapper/120:29:581
26234992515,7cyclictest22243-21sed23:40:141
2623399259,12cyclictest3489-21awk00:14:590
262942240,2sleep00-21swapper/000:07:070
2623699241,1cyclictest0-21swapper/321:05:503
2623699241,1cyclictest0-21swapper/319:29:433
26236992410,9cyclictest52950irq/141-enp3s0-21:20:133
26236992410,9cyclictest52950irq/141-enp3s0-21:20:123
2623699240,12cyclictest100272rm22:19:453
2623499247,10cyclictest22-21ksoftirqd/121:50:321
2623499243,11cyclictest22-21ksoftirqd/122:15:511
2623499242,16cyclictest2395-21kworker/u8:123:05:141
26234992411,8cyclictest53050irq/142-enp3s0-21:35:111
26234992411,3cyclictest141rcu_preempt21:17:391
26236992313,5cyclictest13087-21diskmemload22:50:193
2623599231,17cyclictest18640-21ssh22:50:112
2623499237,4cyclictest22-21ksoftirqd/100:35:381
2623499235,5cyclictest0-21swapper/121:28:451
26234992317,4cyclictest22-21ksoftirqd/122:45:001
26234992311,7cyclictest29726-21apt-get22:10:131
26233992321,1cyclictest0-21swapper/021:05:540
253812230,2sleep00-21swapper/022:55:040
269222220,9sleep334-21ksoftirqd/321:43:523
2623699228,4cyclictest141rcu_preempt00:25:213
2623699222,14cyclictest636-21runrttasks00:15:143
2623699222,14cyclictest636-21runrttasks00:15:133
26236992211,8cyclictest17141-21cron00:25:003
2623699221,17cyclictest27538-21ssh23:20:193
26235992220,1cyclictest0-21swapper/221:06:462
26235992213,7cyclictest16811-21perl22:00:162
26234992215,5cyclictest22-21ksoftirqd/100:20:011
26234992215,5cyclictest22-21ksoftirqd/100:20:001
26233992213,5cyclictest13087-21diskmemload00:35:160
2623399221,16cyclictest640-21snmpd22:00:130
2623699217,4cyclictest141rcu_preempt22:05:013
2623699217,11cyclictest0-21swapper/323:05:183
2623699211,5cyclictest0-21swapper/300:01:183
26236992115,4cyclictest21098-21idleruntime-cro21:40:003
2623699211,10cyclictest31853-21hddtemp_smartct00:35:163
26235992115,4cyclictest30455-21cron22:35:012
26235992114,5cyclictest30270-21sh22:58:262
26235992114,4cyclictest8936-21sh00:18:442
26235992114,4cyclictest8936-21sh00:18:432
2623499218,3cyclictest141rcu_preempt22:55:121
2623499218,2cyclictest141rcu_preempt22:32:381
2623499215,8cyclictest141rcu_preempt21:59:131
2623499213,5cyclictest22-21ksoftirqd/122:38:051
26234992112,4cyclictest22-21ksoftirqd/123:16:591
26234992111,8cyclictest22271-21ssh22:28:331
26234992111,8cyclictest11194-21ssh21:31:501
2623499211,12cyclictest22-21ksoftirqd/100:10:021
2623399219,10cyclictest3509-21sh23:26:070
26233992112,7cyclictest17849-21entropy22:25:150
2623399211,17cyclictest23233-21grep23:40:240
221702210,2sleep00-21swapper/021:15:350
2623699209,7cyclictest52950irq/141-enp3s0-21:55:123
2623699208,9cyclictest0-21swapper/321:10:273
2623699207,4cyclictest1871-21latency23:25:183
2623699201,5cyclictest0-21swapper/321:50:183
26236992012,4cyclictest23729-21latency_hist22:30:013
26236992012,4cyclictest13087-21diskmemload23:10:253
2623699201,17cyclictest21203-21cron23:39:593
2623699201,15cyclictest560-21dbus-daemon00:10:253
26236992011,5cyclictest3644-21missed_timers23:50:203
2623699201,15cyclictest23352-21fschecks_time00:05:143
26236992010,6cyclictest127412chrt22:45:193
2623699200,18cyclictest6192-21cron23:05:013
2623599209,6cyclictest52850irq/140-enp3s0-21:20:122
2623599209,6cyclictest52850irq/140-enp3s0-21:20:112
2623599201,4cyclictest0-21swapper/221:52:452
26235992014,4cyclictest24892-21sh22:05:362
26235992014,4cyclictest23519-21hwlatdetect00:05:172
26235992014,4cyclictest15767-21sh21:10:562
26235992012,6cyclictest2772-21ls21:25:242
26235992012,4cyclictest4955-21needreboot22:15:192
26235992010,6cyclictest52850irq/140-enp3s0-23:45:122
2623599200,8cyclictest133622sh21:57:332
2623599200,18cyclictest11864-21rm23:08:592
2623499207,4cyclictest22-21ksoftirqd/123:54:201
2623499204,2cyclictest141rcu_preempt21:12:141
2623499203,7cyclictest141rcu_preempt22:49:271
2623499202,4cyclictest22-21ksoftirqd/121:22:341
2623499202,4cyclictest22-21ksoftirqd/121:22:331
2623499202,13cyclictest1104-21ssh23:25:131
26234992013,5cyclictest22-21ksoftirqd/100:00:001
26234992012,6cyclictest888-21ntpq23:00:191
26234992012,6cyclictest13716-21entropy23:10:151
2623399207,3cyclictest141rcu_preempt21:10:150
26233992014,3cyclictest10524-21apt-key00:20:000
26233992013,5cyclictest12454-21grep22:45:170
26233992013,4cyclictest29282-21sh23:21:130
26233992013,4cyclictest15688-21sh21:59:520
26233992013,4cyclictest13087-21diskmemload00:28:240
26233992012,6cyclictest4299-21entropy00:15:160
26233992012,6cyclictest4299-21entropy00:15:150
26233992011,7cyclictest560-21dbus-daemon21:35:000
26233992011,6cyclictest560-21dbus-daemon21:55:000
26233992011,6cyclictest15892-21pluginstate21:35:210
2623699195,4cyclictest13087-21diskmemload23:18:323
2623699193,5cyclictest636-21runrttasks22:14:173
2623699191,9cyclictest9053-21ntp_states23:30:223
26236991913,4cyclictest12790-21sh22:20:443
26236991912,5cyclictest24502-21cat00:30:133
26236991911,6cyclictest5669-21tune2fs22:40:153
26236991910,6cyclictest515-21irqbalance22:30:173
26236991910,5cyclictest30099-21grep23:45:263
2623599199,7cyclictest12284-21df_inode22:45:162
2623599198,9cyclictest0-21swapper/221:41:572
2623599196,3cyclictest52850irq/140-enp3s0-22:28:352
2623599191,7cyclictest148892sh23:59:112
26235991914,3cyclictest29519-21sendmail_mailst19:15:232
26235991913,4cyclictest4411-21sh23:27:032
26235991913,4cyclictest2869-21sh00:14:242
26235991913,4cyclictest25550-21sh21:18:582
26235991913,4cyclictest22804-21sh00:28:582
26235991913,4cyclictest15280-21sh23:35:132
26235991911,6cyclictest27094-21cat23:20:142
2623599191,16cyclictest0-21swapper/223:14:182
26235991911,5cyclictest13087-21diskmemload22:37:382
2623499197,4cyclictest22-21ksoftirqd/121:45:221
26234991914,2cyclictest221ksoftirqd/119:15:011
26234991910,2cyclictest141rcu_preempt23:21:471
2623399199,7cyclictest6666-21apt-get23:05:150
2623399193,11cyclictest52750irq/139-enp3s0-22:37:150
2623399192,12cyclictest5196-21ssh22:40:110
2623399191,9cyclictest141rcu_preempt23:35:260
26233991914,3cyclictest2110-21kernelversion21:25:180
26233991913,4cyclictest24494-21sh21:41:250
26233991913,4cyclictest11211-21sh23:31:560
26233991913,3cyclictest24435-21sh22:05:240
26233991912,4cyclictest27123-21sh22:31:390
26233991912,4cyclictest22590-21sh23:16:140
26233991911,6cyclictest30613-21memory22:10:210
26233991910,6cyclictest17183-21seq23:12:330
26236991814,3cyclictest27834-21ssh22:55:583
26236991812,4cyclictest11141-21sh21:31:453
26236991812,3cyclictest32123-21sh22:35:223
2623699181,14cyclictest1980-21ssh21:25:163
2623599187,9cyclictest0-21swapper/222:22:362
2623599184,2cyclictest141rcu_preempt22:42:042
26235991813,3cyclictest52850irq/140-enp3s0-21:45:232
26235991813,3cyclictest2624-21sh23:50:082
26235991813,3cyclictest24786-21sh23:41:502
26235991813,3cyclictest15863-21sh21:35:212
26235991812,4cyclictest7929-21sh21:30:042
26235991812,3cyclictest0-21swapper/223:00:102
26235991811,5cyclictest29834-21ls22:10:152
26235991810,6cyclictest20482-21tune2fs23:15:162
26233991812,3cyclictest7566-21sh22:17:160
26233991810,5cyclictest28611-21cat23:45:130
2623699179,6cyclictest23630-21smartctl22:05:153
2623699178,6cyclictest28892-21gltestperf19:15:153
2623499177,8cyclictest29269-21needreboot19:15:191
2623399179,6cyclictest27658-21ls19:10:260
2623699169,4cyclictest27622-21timerwakeupswit19:10:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional