You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 21:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sun Aug 23, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2956821750,1sleep20-21swapper/211:10:162
87952173102,27sleep00-21swapper/007:05:450
89252172101,64sleep30-21swapper/307:07:293
88892166110,29sleep20-21swapper/207:06:582
88062161109,30sleep10-21swapper/107:05:541
312792730,2sleep00-21swapper/009:59:590
315212670,2sleep10-21swapper/110:00:001
65772650,3sleep06572-21ssh12:05:160
316702650,11sleep012-21ksoftirqd/011:35:220
83272630,1sleep10-21swapper/110:05:581
211012620,3sleep3331rcuc/309:51:323
308772590,2sleep10-21swapper/112:23:141
302612590,1sleep10-21swapper/110:46:111
302612590,1sleep10-21swapper/110:46:111
280512590,1sleep30-21swapper/310:45:133
280512590,1sleep30-21swapper/310:45:133
259842540,3sleep125988-21unixbench_singl09:00:251
237292540,10sleep122-21ksoftirqd/109:30:111
76142520,3sleep3331rcuc/309:41:323
191802510,1sleep30-21swapper/311:25:583
11452490,1sleep10-21swapper/109:36:491
287232450,1sleep30-21swapper/309:57:263
306532440,2chrt22-21ksoftirqd/111:59:231
274882430,1sleep227492-21grep09:05:202
279392410,2sleep00-21swapper/009:32:550
78362400,1sleep00-21swapper/010:05:290
223222400,2chrt0-21swapper/211:29:062
192492380,1sleep10-21swapper/112:15:001
308382350,3sleep20-21swapper/209:59:332
36122340,2chrt0-21swapper/312:26:263
93212330,1chrt0-21swapper/109:43:151
77522330,2sleep2271rcuc/211:41:422
9398993214,10cyclictest34-21ksoftirqd/310:17:003
9398993213,3cyclictest141rcu_preempt09:25:223
9396993213,14cyclictest52750irq/139-enp3s0-10:35:121
163552300,1sleep00-21swapper/012:12:140
939899291,21cyclictest14033-21cstates07:20:153
9398992911,9cyclictest34-21ksoftirqd/311:06:233
9398992911,5cyclictest34-21ksoftirqd/312:05:193
939699294,17cyclictest7548-21ssh10:30:131
939699291,4cyclictest640-21snmpd08:17:141
126162290,2sleep00-21swapper/009:45:230
939799280,25cyclictest242082sleep207:43:502
939699281,25cyclictest0-21swapper/107:59:241
939699281,25cyclictest0-21swapper/107:46:271
939699281,25cyclictest0-21swapper/107:27:521
939599281,25cyclictest0-21swapper/008:31:260
939599281,25cyclictest0-21swapper/007:24:080
65892280,11chrt22-21ksoftirqd/109:16:461
65892280,11chrt22-21ksoftirqd/109:16:461
184842280,2chrt0-21swapper/210:37:542
939899279,4cyclictest34-21ksoftirqd/310:59:013
939899279,4cyclictest34-21ksoftirqd/310:09:253
939899279,4cyclictest34-21ksoftirqd/309:33:003
939899279,10cyclictest34-21ksoftirqd/311:44:373
939899278,5cyclictest34-21ksoftirqd/311:20:013
939899277,5cyclictest34-21ksoftirqd/311:31:463
939899277,5cyclictest34-21ksoftirqd/311:31:463
939899271,24cyclictest0-21swapper/308:18:433
9397992722,3cyclictest515-21irqbalance08:26:572
939799271,24cyclictest0-21swapper/208:50:462
939799271,24cyclictest0-21swapper/208:03:352
9396992718,7cyclictest24566-21munin-run11:55:011
939699271,2cyclictest0-21swapper/108:28:141
939699271,2cyclictest0-21swapper/107:43:251
939699271,25cyclictest0-21swapper/108:53:351
939699271,25cyclictest0-21swapper/108:08:301
939699271,25cyclictest0-21swapper/107:32:531
939699271,24cyclictest0-21swapper/108:55:361
939699271,24cyclictest0-21swapper/108:46:481
939699271,24cyclictest0-21swapper/107:52:081
939699271,24cyclictest0-21swapper/107:35:061
939699271,24cyclictest0-21swapper/107:20:391
9395992725,1cyclictest0-21swapper/007:29:540
939599271,25cyclictest0-21swapper/008:53:120
939599271,25cyclictest0-21swapper/007:32:540
939599271,24cyclictest0-21swapper/009:08:050
939599271,24cyclictest0-21swapper/008:55:080
939599271,24cyclictest0-21swapper/008:42:000
939599271,24cyclictest0-21swapper/008:25:430
939599271,24cyclictest0-21swapper/008:05:110
939599271,24cyclictest0-21swapper/008:00:450
939599271,24cyclictest0-21swapper/007:58:490
939599271,24cyclictest0-21swapper/007:45:310
939599271,24cyclictest0-21swapper/007:40:500
172862270,2sleep30-21swapper/312:13:103
939899268,4cyclictest34-21ksoftirqd/309:14:063
939899266,6cyclictest34-21ksoftirqd/312:20:243
939899261,2cyclictest0-21swapper/309:08:583
939899261,2cyclictest0-21swapper/308:56:013
9398992612,4cyclictest34-21ksoftirqd/311:37:173
939899261,23cyclictest0-21swapper/308:52:353
939899261,23cyclictest0-21swapper/308:32:423
939899261,23cyclictest0-21swapper/308:25:363
939899261,23cyclictest0-21swapper/308:23:473
939899261,23cyclictest0-21swapper/308:08:513
939899261,23cyclictest0-21swapper/308:00:083
939899261,23cyclictest0-21swapper/307:55:473
939899261,23cyclictest0-21swapper/307:53:163
939899261,18cyclictest0-21swapper/307:31:213
9398992611,5cyclictest34-21ksoftirqd/309:49:093
939799262,16cyclictest29038-21ssh09:10:032
9397992614,8cyclictest7109-21cron10:30:012
939799261,23cyclictest0-21swapper/209:00:102
939799261,23cyclictest0-21swapper/208:41:052
939799261,23cyclictest0-21swapper/208:20:032
939799261,23cyclictest0-21swapper/208:07:322
939799261,23cyclictest0-21swapper/207:55:482
939799261,23cyclictest0-21swapper/207:45:552
939799261,23cyclictest0-21swapper/207:35:312
939799261,23cyclictest0-21swapper/207:25:032
939799261,23cyclictest0-21swapper/207:24:162
939799261,18cyclictest0-21swapper/208:30:092
939799260,24cyclictest22810-21cut08:55:162
9396992624,1cyclictest0-21swapper/108:30:371
9396992624,1cyclictest0-21swapper/108:20:001
9396992623,2cyclictest0-21swapper/108:00:511
939699261,24cyclictest0-21swapper/109:08:201
939699261,1cyclictest0-21swapper/108:40:061
939699261,1cyclictest0-21swapper/108:35:391
939699261,1cyclictest0-21swapper/108:10:081
939599263,16cyclictest18290-21ssh09:50:120
9395992624,1cyclictest0-21swapper/008:48:110
9395992624,1cyclictest0-21swapper/008:10:550
9395992612,8cyclictest14305-21apt-get10:35:120
939599261,24cyclictest0-21swapper/009:01:040
939599261,24cyclictest0-21swapper/007:54:180
939599261,23cyclictest0-21swapper/008:35:440
939599261,1cyclictest0-21swapper/008:20:170
939599261,1cyclictest0-21swapper/008:16:440
939599261,1cyclictest0-21swapper/007:36:180
939599260,24cyclictest0-21swapper/007:19:220
110542260,1sleep20-21swapper/212:32:072
939899259,2cyclictest141rcu_preempt12:15:153
939899258,4cyclictest34-21ksoftirqd/311:16:163
939899258,4cyclictest34-21ksoftirqd/310:38:103
9398992523,1cyclictest0-21swapper/308:49:473
9398992523,1cyclictest0-21swapper/308:41:013
9398992520,3cyclictest34-21ksoftirqd/312:00:013
939899251,23cyclictest0-21swapper/307:47:493
939899251,23cyclictest0-21swapper/307:37:223
939899251,1cyclictest0-21swapper/308:36:123
939899251,1cyclictest0-21swapper/307:44:103
939899250,23cyclictest0-21swapper/307:25:583
9397992522,2cyclictest0-21swapper/208:35:062
9397992521,2cyclictest0-21swapper/207:30:262
939799251,23cyclictest0-21swapper/208:17:272
939799251,1cyclictest0-21swapper/207:52:582
9396992523,1cyclictest0-21swapper/107:19:511
211252250,9sleep334-21ksoftirqd/311:51:353
939899249,3cyclictest141rcu_preempt11:01:563
939899248,5cyclictest34-21ksoftirqd/309:37:173
939899248,2cyclictest141rcu_preempt10:13:533
939899247,10cyclictest34-21ksoftirqd/310:01:023
9398992422,1cyclictest0-21swapper/308:11:403
9398992418,4cyclictest34-21ksoftirqd/312:35:003
9397992422,1cyclictest0-21swapper/208:10:092
9397992422,1cyclictest0-21swapper/207:19:532
9397992415,7cyclictest28552-21diskmemload11:09:182
939799240,1cyclictest0-21swapper/208:46:422
939699243,15cyclictest10282-21ssh09:20:121
939699242,15cyclictest699-21taskset12:25:031
939699242,15cyclictest26339-21sh10:20:041
939699242,14cyclictest796-21ssh10:25:121
235362240,9sleep334-21ksoftirqd/310:41:123
123692240,2sleep20-21swapper/210:33:312
939899239,2cyclictest141rcu_preempt10:32:043
939899237,2cyclictest141rcu_preempt10:29:253
939899236,3cyclictest34-21ksoftirqd/312:03:473
9398992321,1cyclictest0-21swapper/307:19:513
9398992317,4cyclictest34-21ksoftirqd/312:40:013
9398992310,4cyclictest34-21ksoftirqd/309:15:533
9398992310,4cyclictest34-21ksoftirqd/309:15:533
9397992313,7cyclictest16950-21grep11:00:242
939799230,3cyclictest18225-21apt-get11:50:122
939699233,17cyclictest7033-21needreboot12:05:201
238702230,2sleep0131rcuc/011:54:210
939899226,4cyclictest34-21ksoftirqd/311:11:383
9398992216,4cyclictest34-21ksoftirqd/309:25:003
939899221,10cyclictest34-21ksoftirqd/310:51:553
939799229,7cyclictest12822-21ssh12:10:122
9397992214,5cyclictest832-21/usr/sbin/munin12:25:252
9397992213,6cyclictest278-21systemd-journal09:50:002
9397992210,7cyclictest53050irq/142-enp3s0-10:50:132
939799220,5cyclictest0-21swapper/210:23:062
939699222,14cyclictest75-21kswapd011:20:111
939599227,4cyclictest66050lldpd11:44:390
9395992215,4cyclictest9578-21kernelversion10:55:180
9395992213,6cyclictest32168-21awk09:35:260
9395992210,8cyclictest14565-21apt-get12:35:110
939899214,8cyclictest141rcu_preempt10:20:023
9398992110,2cyclictest141rcu_preempt11:48:023
9397992112,7cyclictest30842-21diskstats09:35:152
9397992111,8cyclictest15880-21tr12:35:212
939699212,15cyclictest3118-21grep10:50:211
939699212,13cyclictest11531-21apt-get09:45:131
9396992115,4cyclictest22-21ksoftirqd/107:15:011
9396992112,5cyclictest18482-21timerandwakeup09:25:241
9396992110,3cyclictest141rcu_preempt10:10:151
9395992111,6cyclictest52950irq/141-enp3s0-10:30:130
9395992110,6cyclictest0-21swapper/009:25:120
319222210,1sleep10-21swapper/110:00:111
9398992018,1cyclictest0-21swapper/309:00:303
939799208,8cyclictest24956-21apt-get11:55:112
939799207,9cyclictest8812-21apt-get10:55:122
939799202,12cyclictest3571-21ssh09:15:152
939799202,12cyclictest3571-21ssh09:15:152
939799201,6cyclictest11681-21cpuspeed_turbos11:45:152
9397992013,4cyclictest30148-21sh12:22:312
9397992013,4cyclictest28560-21sshd09:33:322
9397992012,6cyclictest20094-21seq10:15:162
9397992012,6cyclictest1745-21cut10:50:002
9397992012,6cyclictest1745-21cut10:50:002
9397992011,6cyclictest5160-21cat09:40:142
9397992011,6cyclictest10345-21ntpq07:10:202
939699202,12cyclictest18263-21apt-get09:50:111
939699202,11cyclictest22341-21ssh11:05:131
939699201,4cyclictest0-21swapper/111:37:271
9396992014,4cyclictest22-21ksoftirqd/112:20:001
9396992012,5cyclictest28552-21diskmemload10:43:031
9396992011,6cyclictest9361-21sh11:43:201
939599209,8cyclictest10654-21diskstats09:20:150
939599209,3cyclictest208432ssh11:03:580
939599206,9cyclictest52950irq/141-enp3s0-10:27:010
939599205,3cyclictest19662ssh10:50:070
939599202,15cyclictest13374-21sed10:10:150
939599202,12cyclictest29229-21ssh09:10:130
9395992013,5cyclictest28851-21ssh11:10:010
9395992013,4cyclictest30489-21sh10:46:260
9395992013,4cyclictest30489-21sh10:46:250
9395992013,4cyclictest27754-21sendmail_mailst10:20:220
9395992013,4cyclictest10715-21gltestperf11:20:160
9395992011,6cyclictest3632-21seq12:26:270
138692200,2chrt0-21swapper/210:10:212
939799192,10cyclictest21068-21seq10:40:122
9397991913,4cyclictest16950-21sh09:25:092
9397991913,3cyclictest4365-21sh12:03:442
9397991912,4cyclictest4983-21sh11:15:262
9397991911,6cyclictest19288-21awk09:50:202
9397991910,7cyclictest6882-21cat12:05:192
9397991910,6cyclictest32252-21perl10:00:152
939699196,5cyclictest14230-21cat12:35:011
939699193,11cyclictest75-21kswapd011:15:141
939699192,13cyclictest29211-21apt-get09:10:131
939699192,12cyclictest75-21kswapd010:15:121
9396991913,4cyclictest23216-21sh11:30:001
9396991913,4cyclictest14533-21/usr/sbin/munin12:35:141
9396991913,3cyclictest14417-21sh10:59:191
9396991911,6cyclictest24690-21cat11:30:201
9396991911,6cyclictest24690-21cat11:30:201
939699191,15cyclictest22082-21latency_hist11:05:011
9396991910,6cyclictest278-21systemd-journal11:50:001
939599192,11cyclictest7658-21apt-get12:30:140
9395991913,4cyclictest6412-21sh09:16:360
9395991913,4cyclictest6412-21sh09:16:360
9395991913,4cyclictest636-21runrttasks12:03:510
9395991913,4cyclictest29439-21sh11:58:100
9395991912,5cyclictest32386-21cat10:00:160
9395991912,5cyclictest23451-21seq12:17:360
9395991912,5cyclictest17645-21awk11:25:170
9395991912,4cyclictest4591-21sh09:40:020
9395991912,4cyclictest25579-21sh10:43:150
257292190,1sleep20-21swapper/211:30:472
257292190,1sleep20-21swapper/211:30:472
939799182,10cyclictest19554-21apt-get12:15:132
9397991813,3cyclictest9094-21sh10:06:442
9397991812,4cyclictest12101-21sh09:20:322
9397991812,3cyclictest9955-21sh11:20:022
9396991812,4cyclictest28552-21diskmemload12:01:491
9396991810,5cyclictest624-21gdbus11:15:011
939599184,8cyclictest26322-21ssh12:20:120
939599182,2cyclictest141rcu_preempt11:10:000
939599182,12cyclictest19525-21apt-get10:15:130
9395991813,3cyclictest3314-21sh11:15:110
9395991812,4cyclictest13736-21sh11:45:580
939599181,12cyclictest640-21snmpd11:30:120
939599181,12cyclictest640-21snmpd11:30:120
9398991711,4cyclictest34-21ksoftirqd/307:15:003
939799177,6cyclictest53050irq/142-enp3s0-11:35:122
9395991711,4cyclictest11137-21seq07:12:460
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional