You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 06:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackaslot8s.osadl.org (updated Mon May 04, 2026 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
121952220100,96sleep10-21swapper/119:06:121
124252177101,25sleep30-21swapper/319:09:113
12479217197,25sleep20-21swapper/219:09:542
12229215695,26sleep00-21swapper/019:06:390
44542950,3sleep20-21swapper/223:13:312
3662790,2sleep10-21swapper/122:45:411
82572750,1chrt0-21swapper/100:28:381
8392630,1sleep30-21swapper/300:22:553
141672610,3sleep122-21ksoftirqd/123:45:001
174122580,2sleep20-21swapper/223:22:462
155072580,3sleep1211rcuc/122:33:331
276112560,2sleep30-21swapper/321:00:013
187072560,1sleep30-21swapper/323:24:033
181902560,1sleep00-21swapper/023:23:330
50322550,2sleep30-21swapper/322:01:053
250662550,1sleep30-21swapper/321:28:423
107992550,1sleep20-21swapper/221:17:532
79332520,2sleep20-21swapper/221:40:142
161452510,1sleep00-21swapper/000:34:520
12760994914,13cyclictest25758-21/usr/sbin/munin23:05:150
3142480,3sleep2317-21gltestperf19:55:152
278142480,2sleep10-21swapper/121:30:201
205082480,1sleep120510-21sh23:49:421
70002470,1sleep10-21swapper/122:26:471
140672470,3chrt12758-21cyclictest22:08:233
85712460,1sleep10-21swapper/122:52:071
83112460,2sleep228-21ksoftirqd/222:51:522
12760994416,13cyclictest22118-21rm22:38:270
12760994414,15cyclictest28522-21sed19:45:170
60782430,1sleep30-21swapper/300:02:323
187412430,4sleep018712-21ssh23:00:140
12760994316,13cyclictest25991-21ssh21:29:370
12760994216,10cyclictest5294-21rm00:01:440
12760994213,11cyclictest2330-21/usr/sbin/munin21:35:240
12760994212,16cyclictest0-21swapper/022:32:020
12760994116,16cyclictest32604-21ssh21:58:110
12760994114,9cyclictest2354-21apt-config22:00:000
12760994114,11cyclictest16042-21sed21:45:230
12760994114,10cyclictest7513-21grep23:15:200
12760994016,9cyclictest2558-21ssh23:11:380
12760994016,15cyclictest9418-21ssh00:29:490
12760994016,14cyclictest21161-21rm23:50:070
12760994015,7cyclictest24835-21sh22:15:390
12760994015,16cyclictest21626-21ssh00:38:370
12760994014,9cyclictest31814-21diskmemload21:44:090
12760994014,9cyclictest23687-21sensors00:15:240
12760994014,9cyclictest23687-21sensors00:15:230
12760994013,9cyclictest4977-21ssh22:50:030
12760994013,10cyclictest16322-21ssh23:45:290
1276199392,9cyclictest22330-21apt-get00:15:121
1276199392,9cyclictest22330-21apt-get00:15:121
1276099398,19cyclictest0-21swapper/022:26:510
12760993914,9cyclictest26286-21ssh21:53:360
12760993914,9cyclictest2259-21ssh23:35:230
12760993914,9cyclictest2259-21ssh23:35:220
12760993914,14cyclictest13612-21ssh22:55:260
12760993913,8cyclictest10415-21ssh21:17:310
12760993913,10cyclictest21935-21chrt23:25:330
12760993912,9cyclictest26197-21ssh22:40:470
12760993910,8cyclictest0-21swapper/023:40:240
12760993816,8cyclictest21517-21ssh22:14:070
12760993815,9cyclictest31814-21diskmemload21:24:150
12760993813,9cyclictest9390-21dpkg22:05:110
12760993813,9cyclictest21418-21cat20:45:130
12760993813,10cyclictest32623-21rm21:10:140
12760993811,10cyclictest25600-21/usr/sbin/munin20:55:180
1276099377,18cyclictest0-21swapper/000:10:110
12760993715,8cyclictest31814-21diskmemload21:30:540
12760993713,9cyclictest28181-21switchtime23:30:230
12760993713,8cyclictest1221-21sed19:55:240
12760993712,9cyclictest29902-21/usr/sbin/munin21:05:130
12760993615,7cyclictest8514-21/usr/sbin/munin20:15:130
12760993613,9cyclictest11283-21ssh00:06:030
12760993613,10cyclictest730-21ssh22:46:010
12760993613,10cyclictest32556-21ssh00:22:120
12760993613,10cyclictest25343-21seq20:54:480
12760993613,10cyclictest18693-21seq20:37:030
12760993612,9cyclictest29935-21/usr/sbin/munin19:50:200
12760993611,14cyclictest0-21swapper/023:56:070
12760993516,17cyclictest15450-21tail20:30:160
12760993512,8cyclictest4549-21fschecks_count20:05:130
12760993512,7cyclictest31228-21unixbench_multi22:20:230
263872330,2chrt0-21swapper/200:17:442
263872330,2chrt0-21swapper/200:17:442
1276099338,15cyclictest0-21swapper/019:35:150
12760993314,8cyclictest16643-21seq19:17:420
12760993313,8cyclictest14491-21seq19:12:360
12762993218,9cyclictest53050irq/142-enp3s0-22:55:122
76882310,1sleep30-21swapper/320:10:253
12760993118,10cyclictest18485-21/usr/sbin/munin19:20:240
12760993115,13cyclictest19932-21ls20:40:170
12760993020,7cyclictest21530-21tr19:30:130
12760993017,11cyclictest28130-21/usr/sbin/munin21:00:130
12760993016,11cyclictest27278-21taskset19:41:240
77602290,1sleep20-21swapper/222:03:482
203522290,3chrt0-21swapper/100:13:251
12760992919,8cyclictest19230-21/usr/sbin/munin19:25:110
12760992918,9cyclictest8245-21latency_hist20:15:000
12760992918,8cyclictest2129-21/usr/sbin/munin20:00:130
119642290,1sleep111965-21cpuspeed_turbos22:55:121
12760992815,11cyclictest13705-21awk20:25:200
1276199271,24cyclictest0-21swapper/119:12:501
12763992618,6cyclictest34-21ksoftirqd/322:00:003
12762992624,1cyclictest0-21swapper/219:12:502
12760992621,2cyclictest10628-21apt-get20:20:120
325202250,1sleep30-21swapper/322:21:353
1276399251,1cyclictest0-21swapper/319:12:343
1276299253,16cyclictest3924-21ssh22:25:112
1276199253,14cyclictest22460-21ssh22:15:001
19682240,3sleep228-21ksoftirqd/200:24:022
12763992414,7cyclictest11042-21df_inode22:30:153
1276299247,5cyclictest0-21swapper/223:02:292
12762992413,8cyclictest7435-21basename23:40:002
12762992413,8cyclictest7435-21basename23:40:002
1276199243,15cyclictest29619-21sshd22:20:101
88982230,2sleep30-21swapper/323:16:013
267412230,4sleep2271rcuc/222:17:332
131692230,1sleep10-21swapper/122:07:291
12762992312,8cyclictest15107-21grep23:45:162
1276299230,5cyclictest0-21swapper/221:54:462
12761992312,8cyclictest26797-21apt-get23:30:101
12761992311,6cyclictest31814-21diskmemload23:00:121
121892231,3sleep312208-21ssh00:06:593
12763992214,6cyclictest25030-21users19:35:263
12763992212,7cyclictest1673-21fw_packets21:35:183
12763992211,8cyclictest34-21ksoftirqd/321:10:123
1276299224,10cyclictest281ksoftirqd/221:38:052
1276299222,15cyclictest2417-1kworker/u9:222:35:132
12761992213,7cyclictest560-21dbus-daemon20:00:001
1276199221,18cyclictest25784-21grep19:40:131
1276199221,10cyclictest10029-21apt-get00:30:111
12761992210,8cyclictest29960-21apt-get19:50:111
58712210,2sleep30-21swapper/323:38:283
58712210,2sleep30-21swapper/323:38:283
187182210,1sleep30-21swapper/300:11:473
1276399217,11cyclictest19334-21cut19:25:133
12763992113,6cyclictest25495-21ssh22:16:183
12763992112,7cyclictest21806-21tune2fs19:30:153
12763992112,7cyclictest15706-21awk21:45:213
12763992112,6cyclictest20105-21ntpq20:40:193
1276299214,4cyclictest0-21swapper/222:13:162
1276299214,11cyclictest515-21irqbalance00:10:132
1276299213,11cyclictest14648-21ssh21:45:122
1276299212,13cyclictest21351-21apt-get20:45:132
1276299212,13cyclictest2055-21apt-get20:00:132
1276299211,5cyclictest0-21swapper/219:15:242
12762992112,7cyclictest31081-21sensors_temp21:05:252
12762992112,4cyclictest28-21ksoftirqd/223:55:112
1276199219,3cyclictest133722sh21:20:101
1276199217,5cyclictest10478-21timerwakeupswit00:05:251
12761992113,6cyclictest6285-21latency_hist20:10:011
12761992110,6cyclictest52850irq/140-enp3s0-00:20:121
1276199210,8cyclictest270050tune2fs00:00:151
283232200,2sleep10-21swapper/119:45:151
12763992014,4cyclictest27697-21missed_timers23:30:183
12763992012,6cyclictest8817-21awk20:15:143
12763992012,6cyclictest30263-21diskstats21:05:173
12763992012,6cyclictest27734-21sed21:30:203
12763992012,6cyclictest17419-21expr19:20:133
12763992012,6cyclictest13523-21ls20:25:173
12763992011,8cyclictest3448-21ssh23:12:303
12763992011,5cyclictest34-21ksoftirqd/319:55:033
12763992010,7cyclictest5341-21sensors_temp20:05:203
1276299208,8cyclictest26530-21wc23:05:232
1276299207,9cyclictest28-21ksoftirqd/222:05:122
1276299207,11cyclictest3322-21sshd21:12:102
1276299202,13cyclictest45-21kcompactd000:35:132
12762992013,5cyclictest15829-21sh20:30:202
12762992011,6cyclictest5633-21awk20:05:232
1276199206,12cyclictest8106-21ssh21:40:131
1276199202,13cyclictest13346-21ssh23:20:121
12761992013,5cyclictest636-21runrttasks23:59:261
12761992012,6cyclictest6606-21apt-get23:15:121
12761992012,6cyclictest21098-21ntp_states21:25:201
12761992011,6cyclictest21720-21diskstats19:30:141
1276199201,16cyclictest1984-21cron22:00:001
12761992010,6cyclictest25637-21ls22:40:251
1276199200,4cyclictest0-21swapper/121:52:191
228532190,2sleep10-21swapper/120:45:261
1276399192,12cyclictest27825-21apt-get19:45:123
1276399191,6cyclictest266812sh21:53:593
12763991914,3cyclictest14128-21sh23:45:003
12763991913,4cyclictest21068-21sh23:50:013
12763991913,3cyclictest15905-21switchtime23:45:233
12763991912,5cyclictest997-21seq23:59:123
12763991912,5cyclictest30410-21sh22:45:003
12763991912,5cyclictest10890-21expr20:20:143
12763991912,4cyclictest18961-21sendmail20:40:003
12763991911,6cyclictest26023-21diskstats19:40:153
12763991911,6cyclictest21479-21cat23:25:233
12763991911,6cyclictest20900-21seq22:13:303
12763991911,6cyclictest16385-21grep20:30:253
12763991911,6cyclictest13608-21ls21:20:123
1276399191,15cyclictest10172-21tr00:30:133
12763991910,6cyclictest21537-21expr20:45:143
1276299197,10cyclictest28-21ksoftirqd/200:05:232
1276299193,4cyclictest0-21swapper/222:32:132
1276299192,10cyclictest640-21snmpd22:40:132
12762991913,4cyclictest28-21ksoftirqd/219:50:002
12762991912,5cyclictest27971-21awk21:30:232
12762991911,6cyclictest30536-21date22:20:172
1276299191,15cyclictest32024-21ssh22:45:222
12762991910,7cyclictest29661-21cron21:05:002
1276299190,5cyclictest0-21swapper/220:25:162
1276199199,6cyclictest52850irq/140-enp3s0-21:15:111
1276199192,12cyclictest21277-21kworker/u8:123:05:131
12761991914,3cyclictest18640-21/usr/sbin/munin22:35:231
12761991914,3cyclictest1682-21gltestperf23:35:171
12761991914,3cyclictest1682-21gltestperf23:35:171
12761991912,5cyclictest14907-21sed19:15:011
12761991911,5cyclictest278-21systemd-journal21:05:001
1276199191,13cyclictest1168-21taskset21:35:141
12761991910,7cyclictest560-21dbus-daemon22:05:001
1276399188,8cyclictest13021-21cat21:20:003
1276399182,12cyclictest27782-21apt-get21:00:113
1276399181,8cyclictest7567-21taskset22:51:073
12763991813,3cyclictest29944-21acpi19:50:113
12763991813,3cyclictest26644-21sendmail_mailtr23:05:233
12763991813,3cyclictest23866-21fschecks_count20:50:153
12763991813,3cyclictest22696-21sh23:02:473
12763991812,4cyclictest18950irq/143-ahci[0022:35:153
12763991811,5cyclictest8065-21cat21:40:133
12763991811,5cyclictest5321-21date22:25:223
12763991811,5cyclictest32116-21run-parts22:45:233
12763991811,5cyclictest23262-21sed00:15:193
12763991811,5cyclictest23262-21sed00:15:193
1276399181,15cyclictest12240-21diskstats22:55:143
12763991811,4cyclictest3386-21cpu00:25:123
12763991810,6cyclictest16972-21expr00:35:143
12763991810,5cyclictest2746-21latency20:00:183
1276299189,6cyclictest3318-21acpi00:25:102
1276299185,3cyclictest141rcu_preempt21:25:112
1276299182,12cyclictest23527-21apt-get19:35:132
1276299181,4cyclictest0-21swapper/223:29:042
12762991812,4cyclictest28-21ksoftirqd/200:05:002
12762991812,4cyclictest24850-21sh23:52:172
12762991811,5cyclictest24488-21which20:50:222
12762991811,5cyclictest21143-21date20:45:002
12762991811,5cyclictest17298-21expr19:20:132
12762991811,5cyclictest17287-21expr20:35:142
1276299181,14cyclictest11044-21ssh23:18:102
1276299181,12cyclictest14623-21ssh21:20:212
12762991810,6cyclictest19763-21expr19:25:152
1276199188,8cyclictest32409-21df23:10:131
1276199187,3cyclictest141rcu_preempt19:40:001
12761991813,3cyclictest20133-21ntp_states20:40:201
12761991813,3cyclictest13338-21hwlatdetect20:25:161
12761991812,4cyclictest18411-21unixbench_singl00:35:261
12761991811,5cyclictest23330-21fschecks_time22:15:141
12761991811,5cyclictest14873-21expr21:45:131
12761991810,6cyclictest25498-21date20:55:011
12761991810,6cyclictest16143-21cat20:30:231
1276299173,10cyclictest25628-21apt-get20:55:122
1276299172,9cyclictest28161-21ssh21:55:132
1276299172,10cyclictest8466-21apt-get20:15:122
1276299172,10cyclictest7861-21apt-get23:40:132
12762991712,3cyclictest14819-21sh00:33:312
12762991711,4cyclictest28951-21sh23:30:502
12762991710,5cyclictest7585-21cat20:10:252
12762991710,5cyclictest25881-21expr19:40:142
1276199172,11cyclictest20066-21apt-get23:25:121
12761991712,3cyclictest4212-21sh21:13:041
12761991711,4cyclictest23682-21sh23:51:081
12761991710,5cyclictest9169-21sed20:15:191
12763991610,4cyclictest16463-21vmstat19:15:263
1276299168,6cyclictest30703-21sed19:50:172
1276199169,5cyclictest3998-21sed20:05:011
1276199168,6cyclictest31214-21sed21:05:261
1276199168,6cyclictest10604-21acpi20:20:111
12761991612,2cyclictest6305-21/usr/sbin/munin20:10:231
12761991611,3cyclictest19221-21/usr/sbin/munin19:25:121
12761991610,4cyclictest18447-21timerwakeupswit19:20:241
1276299156,6cyclictest21892-21gltestperf19:30:162
1276199159,4cyclictest18510-21unixbench_singl20:35:261
1276199151,12cyclictest25760-21tr20:55:121
1276299148,4cyclictest636-21runrttasks20:20:092
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional