You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-23 - 11:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Jun 23, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25936217197,25sleep00-21swapper/019:09:320
256272143106,25sleep10-21swapper/119:05:331
257172141106,24sleep20-21swapper/219:06:412
256342138101,26sleep30-21swapper/319:05:383
57242690,2sleep10-21swapper/123:53:321
57242690,2sleep10-21swapper/123:53:311
86632590,1sleep00-21swapper/023:30:460
131962590,2sleep30-21swapper/323:10:203
95762550,1sleep10-21swapper/122:43:401
158532530,2sleep20-21swapper/220:00:142
197852520,1sleep00-21swapper/000:27:540
9012490,1sleep20-21swapper/221:25:012
83572490,1sleep30-21swapper/319:40:243
158592490,1sleep20-21swapper/222:48:162
191422460,2sleep20-21swapper/200:03:202
320682430,3sleep232065-21users22:35:282
219382430,1sleep321937-21ssh23:16:423
159402430,1sleep00-21swapper/022:00:140
118102300,5sleep334-21ksoftirqd/321:56:193
270312290,10sleep122-21ksoftirqd/100:33:291
67922280,8sleep334-21ksoftirqd/323:05:223
282332280,2sleep20-21swapper/200:34:422
26248992816,10cyclictest5852-21rm23:53:402
26248992816,10cyclictest5852-21rm23:53:402
12412280,2chrt0-21swapper/220:40:172
174032270,3chrt0-21swapper/322:25:123
163922270,3sleep126244-21cyclictest21:11:351
26247992616,7cyclictest1501-21ssh00:14:451
26247992612,6cyclictest10236-21open_files19:45:201
26246992613,11cyclictest14112-21latency_hist00:00:000
26249992515,8cyclictest14127-21tr00:00:003
2624999251,1cyclictest0-21swapper/319:15:073
26249992510,8cyclictest1176-21ssh21:25:113
26248992518,5cyclictest15459-21sh22:00:012
2624799253,4cyclictest22-21ksoftirqd/119:10:221
228762250,1sleep20-21swapper/222:29:422
208662250,4sleep110582-21kworker/u8:021:39:591
2624999241,1cyclictest0-21swapper/319:14:413
26248992422,1cyclictest0-21swapper/219:15:332
2624899240,18cyclictest0-21swapper/219:15:002
26246992422,1cyclictest0-21swapper/019:15:000
26246992410,4cyclictest66050lldpd21:13:450
261412240,2sleep30-21swapper/300:32:343
112052240,3sleep122-21ksoftirqd/123:33:201
54252230,3sleep30-21swapper/323:53:143
54252230,3sleep30-21swapper/323:53:143
34482230,3sleep20-21swapper/223:27:152
2624999237,6cyclictest0-21swapper/322:50:203
2624999232,15cyclictest30331-21ssh22:35:113
2624899239,12cyclictest3452-21missed_timers19:30:192
2624899233,17cyclictest20501-21grep23:15:242
26248992314,6cyclictest26030-21python322:55:222
26248992313,8cyclictest17331-21users00:25:252
2624799239,10cyclictest29004-21ssh00:35:131
2624799239,10cyclictest29004-21ssh00:35:131
26247992321,1cyclictest0-21swapper/119:15:341
26247992314,7cyclictest13614-21python323:10:231
26247992313,7cyclictest560-21dbus-daemon23:25:001
26246992314,7cyclictest9644-21diskstats19:45:150
26246992313,8cyclictest32316-21apt-get23:25:110
26246992313,7cyclictest278-21systemd-journal22:45:010
2624999222,15cyclictest13296-21ssh23:35:133
2624999222,14cyclictest15627-21ssh00:25:133
26249992215,5cyclictest832-21ssh00:14:053
26249992213,5cyclictest53050irq/142-enp3s0-22:00:143
26248992213,6cyclictest28501-21head00:10:162
26248992213,6cyclictest16975-21ntpq22:00:212
26248992212,7cyclictest6107-21cat19:35:242
2624899221,17cyclictest26226-21sed23:20:162
26248992210,8cyclictest17883-21apt-get20:05:122
2624799224,4cyclictest141rcu_preempt00:05:121
2624799223,13cyclictest27971-21taskset21:45:091
2624799222,5cyclictest21261-21apt-get21:40:121
2624799221,6cyclictest0-21swapper/123:05:261
2624799220,4cyclictest0-21swapper/122:32:361
2624799220,13cyclictest17869-21ssh22:25:171
2624699227,5cyclictest636-21runrttasks22:10:290
2624699221,9cyclictest4018-21cut22:15:150
26246992215,4cyclictest21247-21sh00:05:120
26246992213,7cyclictest30317-21python300:35:240
26246992213,7cyclictest30317-21python300:35:230
112532220,3sleep334-21ksoftirqd/321:05:153
2624999213,11cyclictest24713-21ssh22:55:133
2624999212,13cyclictest22635-21ssh22:05:133
26249992112,6cyclictest10411-21expr00:20:233
26249992110,8cyclictest32426-21ntpq23:00:223
2624999210,7cyclictest27102-21ssh00:09:383
2624899217,3cyclictest303612cat22:10:202
26248992115,4cyclictest28-21ksoftirqd/221:40:002
26248992113,6cyclictest9206-21python321:30:232
26248992113,6cyclictest545-21awk19:25:112
2624899211,17cyclictest14117-21munin-run00:00:002
2624799214,4cyclictest13087-21diskmemload00:00:191
2624799212,6cyclictest2694-21perl00:15:161
26247992113,6cyclictest21093-21cut20:10:231
26247992113,6cyclictest12737-21seq21:05:431
26247992111,5cyclictest24283-21grep22:05:261
2624799211,14cyclictest26939-21sh21:20:121
2624699218,9cyclictest6901-21apt-get20:55:130
26246992119,1cyclictest0-21swapper/019:15:000
26246992113,6cyclictest13684-21seq21:34:330
26246992113,6cyclictest12681-21ls19:50:230
26246992112,7cyclictest6615-21seq20:54:440
26246992112,7cyclictest26708-21seq22:07:480
26246992112,7cyclictest1239-21memory20:40:180
26246992111,7cyclictest9039-21cat21:00:110
26246992111,7cyclictest1285-21df23:50:140
26246992111,7cyclictest1285-21df23:50:130
2624999208,10cyclictest13087-21diskmemload22:18:103
26249992013,5cyclictest8705-21sed00:20:013
26249992012,6cyclictest12840-21seq22:21:173
26249992012,5cyclictest8804-21awk21:00:013
26249992011,6cyclictest3815-21cat20:45:243
26249992011,6cyclictest3815-21cat20:45:233
2624999201,13cyclictest19997-21chrt23:40:133
26249992010,6cyclictest13087-21diskmemload23:45:243
26249992010,6cyclictest11518-21seq22:45:133
2624899208,6cyclictest2725-21fw_packets00:15:152
2624899204,3cyclictest13087-21diskmemload23:00:072
2624899202,6cyclictest28-21ksoftirqd/200:24:592
26248992014,4cyclictest26211-21sh23:45:002
26248992014,4cyclictest21190-21/usr/sbin/munin00:05:182
26248992014,3cyclictest23200-21gltestperf22:05:182
26248992012,6cyclictest32407-21seq19:24:162
26248992012,6cyclictest24652-21df20:20:132
26248992012,6cyclictest13979-21cut19:55:142
2624799208,10cyclictest52950irq/141-enp3s0-00:20:221
2624799204,3cyclictest141rcu_preempt22:50:011
2624799203,5cyclictest22-21ksoftirqd/120:55:111
2624799203,3cyclictest13087-21diskmemload21:55:351
2624799202,16cyclictest20563-21unixbench-2d23:15:261
2624799201,9cyclictest4515-21perf20:50:001
2624799201,9cyclictest4515-21perf20:49:591
26247992014,4cyclictest26645-21sh23:45:111
26247992013,4cyclictest13087-21diskmemload23:25:461
26247992013,4cyclictest0-21swapper/100:25:121
26247992012,6cyclictest25954-21seq20:21:551
26247992012,6cyclictest2429-21basename19:30:011
26247992012,5cyclictest52950irq/141-enp3s0-22:10:131
2624799201,16cyclictest31637-21grep20:35:181
2624699209,7cyclictest52750irq/139-enp3s0-00:10:250
2624699202,12cyclictest21281-21sh21:40:100
26246992014,4cyclictest19726-21sh22:50:250
26246992013,5cyclictest636-21runrttasks21:55:100
26246992013,5cyclictest20294-21swap23:15:230
26246992012,6cyclictest30782-21cat19:20:130
26246992012,6cyclictest29842-21ls20:30:230
26246992011,7cyclictest9321-21tune2fs00:20:140
26246992011,7cyclictest12720-21grep23:10:160
26246992011,6cyclictest7549-21/usr/sbin/munin19:40:150
26246992011,5cyclictest52750irq/139-enp3s0-21:45:090
2624699201,13cyclictest2326-21ssh21:50:040
26246992010,7cyclictest27042-21cat23:45:150
146952200,3chrt13087-21diskmemload21:35:140
2624999197,10cyclictest0-21swapper/321:45:183
2624999193,9cyclictest14563-21apt-get21:35:123
26249991913,3cyclictest636-21runrttasks20:05:133
26249991912,5cyclictest5159-21tune2fs20:50:143
26249991911,6cyclictest6188-21grep19:35:263
26249991911,6cyclictest11699-21expr19:50:133
26249991911,5cyclictest738-21cron21:24:593
26249991911,5cyclictest23000-21grep20:15:183
26249991911,5cyclictest13087-21diskmemload22:34:503
2624899198,8cyclictest22514-21kworker/u8:223:05:122
2624899197,4cyclictest52850irq/140-enp3s0-23:10:442
2624899197,4cyclictest52850irq/140-enp3s0-21:47:542
2624899195,3cyclictest206522grep21:15:142
2624899193,4cyclictest28-21ksoftirqd/221:51:082
2624899192,3cyclictest141rcu_preempt21:25:112
2624899191,8cyclictest4636-21ssh22:40:122
26248991914,3cyclictest9667-21sh23:31:482
26248991914,3cyclictest20013-21sh22:50:412
26248991913,4cyclictest7180-21sh22:17:202
26248991913,4cyclictest32232-21sh23:49:192
26248991913,4cyclictest28904-21sh00:35:092
26248991913,4cyclictest28904-21sh00:35:082
26248991913,4cyclictest27171-21sh22:32:142
26248991911,6cyclictest3757-21run-parts20:45:232
26248991911,6cyclictest3757-21run-parts20:45:222
2624899190,5cyclictest11852-21ls22:20:242
2624799194,3cyclictest141rcu_preempt21:15:121
2624799193,4cyclictest22-21ksoftirqd/121:25:131
2624799192,6cyclictest18017-21apt-get22:50:111
2624799192,11cyclictest10281-21diskmemload22:20:121
2624799191,9cyclictest0-21swapper/123:39:371
2624799191,5cyclictest0-21swapper/123:40:141
2624799191,4cyclictest0-21swapper/121:30:241
26247991912,5cyclictest25180-21cut22:55:161
26247991912,4cyclictest32061-21sh22:35:271
26247991911,6cyclictest19806-21sed20:10:001
2624799191,15cyclictest5339-21cut19:35:171
2624799191,13cyclictest3819-21sh21:50:241
2624799190,7cyclictest32547-21taskset23:00:231
2624699199,6cyclictest14660-21cpuspeed_turbos00:00:120
2624699196,9cyclictest648-21ntpd22:35:210
2624699194,14cyclictest17995-21cut20:05:140
26246991914,3cyclictest4963-21sh00:16:290
26246991913,4cyclictest4662-21sh21:27:130
26246991913,4cyclictest2142-21sh23:01:530
26246991913,4cyclictest15959-21sh23:36:250
26246991912,5cyclictest14539-21ls19:55:210
26246991911,6cyclictest20809-21date23:40:180
26246991910,6cyclictest30907-21cat20:35:120
206502190,1sleep00-21swapper/021:15:150
2624999188,2cyclictest141rcu_preempt00:35:143
2624999188,2cyclictest141rcu_preempt00:35:133
2624999187,7cyclictest53050irq/142-enp3s0-00:00:123
2624999182,4cyclictest2476-21apt-get21:50:133
2624999182,11cyclictest24324-21apt-get20:20:133
2624999182,10cyclictest30591-21apt-get19:20:113
26249991811,5cyclictest6082-21cat22:40:243
26249991811,5cyclictest29871-21cat20:30:253
26249991811,5cyclictest2856-21expr19:30:123
26249991811,5cyclictest1264-21sed20:40:183
26249991811,4cyclictest13087-21diskmemload23:27:363
26249991810,6cyclictest20093-21cat20:10:133
26249991810,5cyclictest22960-21head21:40:253
2624899189,7cyclictest7766-21cat19:40:192
2624899186,4cyclictest53050irq/142-enp3s0-19:45:222
26248991813,3cyclictest9034-21/usr/sbin/munin21:00:182
26248991813,3cyclictest16440-21sh21:11:382
26248991813,3cyclictest15593-21sh23:36:022
26248991812,4cyclictest23609-21sh21:41:012
26248991811,5cyclictest27240-21ls20:25:202
26248991811,5cyclictest24248-21wc20:20:012
26248991811,5cyclictest11185-21find21:05:122
26248991810,6cyclictest31025-21cat20:35:132
26248991810,6cyclictest11945-21expr19:50:162
26248991810,5cyclictest5271-21grep20:50:152
2624799188,7cyclictest804-21df20:40:131
26247991811,5cyclictest12599-21sensors19:50:231
2624799181,14cyclictest16594-21cut20:00:201
2624799181,13cyclictest16981-21sshd22:00:221
2624799181,12cyclictest7661-21apt-get23:55:041
26247991810,6cyclictest7014-21awk19:40:121
26247991810,6cyclictest30354-21seq20:33:221
2624699182,11cyclictest10262-21apt-get22:20:120
2624699181,8cyclictest24634-21ssh22:55:080
26246991813,3cyclictest10743-21sh23:08:540
26246991812,4cyclictest26514-21missed_timers23:20:190
26246991812,4cyclictest17945-21ntp_offset22:25:170
26246991811,5cyclictest2830-21cat19:30:130
26246991811,5cyclictest26648-21cat20:25:140
26246991811,5cyclictest25150-21needreboot20:20:180
26246991811,5cyclictest12597-21readlink22:45:230
26246991811,4cyclictest26775-21sh21:20:020
26246991810,6cyclictest22845-21expr00:30:170
26246991810,6cyclictest16064-21expr20:00:130
26246991810,5cyclictest1497-21pmu-power19:25:200
2624999179,6cyclictest26903-21tune2fs20:25:153
2624999179,4cyclictest26414-21kernelversion23:20:173
2624999178,7cyclictest21365-21pmu-power21:15:223
2624999178,6cyclictest53050irq/142-enp3s0-21:30:123
26249991712,3cyclictest636-21runrttasks19:55:143
26249991711,4cyclictest96352sleep319:45:143
26249991711,4cyclictest31474-21sed20:35:163
26249991711,4cyclictest13087-21diskmemload22:11:333
26249991711,4cyclictest1095-21idleruntime19:25:163
26249991711,3cyclictest13087-21diskmemload21:10:483
26249991710,4cyclictest13087-21diskmemload23:30:323
2624899179,6cyclictest7722-21date20:55:192
2624899172,11cyclictest28606-21apt-get20:30:122
2624799178,7cyclictest1-21systemd19:35:001
2624799177,8cyclictest52950irq/141-enp3s0-22:15:191
2624799176,2cyclictest22-21ksoftirqd/121:00:181
2624799172,13cyclictest27373-21ntp_kernel_pll_20:25:201
26247991712,3cyclictest22813-21kernelversion20:15:161
26247991710,5cyclictest6200-21awk20:50:251
26247991710,5cyclictest278-21systemd-journal19:59:591
2624699179,6cyclictest3370-21date20:45:190
2624699179,6cyclictest3370-21date20:45:180
2624699179,6cyclictest24121-21tune2fs22:30:160
2624699172,10cyclictest11199-21apt-get21:05:150
26246991712,3cyclictest5130-21sh19:35:150
26246991712,3cyclictest23107-21open_inodes20:15:200
26246991710,5cyclictest20248-21expr20:10:140
2624999168,6cyclictest16979-21date20:00:243
26249991612,2cyclictest9060-21apt-get21:00:113
2624899161,4cyclictest28-21ksoftirqd/220:10:112
26247991611,3cyclictest32481-21latency_hist19:25:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional