You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-24 - 16:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Dec 24, 2025 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1755029683,9sleep00-21swapper/007:09:550
1726328962,9sleep30-21swapper/307:06:123
1740628768,9sleep20-21swapper/207:08:042
1747228263,15sleep10-21swapper/107:08:551
1763699210,20cyclictest0-21swapper/010:45:120
1764599191,16cyclictest0-21swapper/211:45:152
1764399190,17cyclictest0-21swapper/108:50:131
58962180,0sleep20-21swapper/211:15:132
17643991817,1cyclictest0-21swapper/108:15:141
1763699186,6cyclictest24788-21perf09:40:000
1764999172,14cyclictest0-21swapper/307:40:133
17643991716,0cyclictest0-21swapper/111:10:131
1763699171,13cyclictest28640-21apt-get08:40:140
17649991612,3cyclictest39-21ksoftirqd/308:40:193
1764999160,3cyclictest0-21swapper/312:15:123
1764399167,1cyclictest141rcu_preempt07:50:251
17643991615,1cyclictest0-21swapper/108:00:231
1763699164,6cyclictest5641-21idleruntime-cro11:14:590
1763699162,8cyclictest0-21swapper/010:25:130
1764999157,5cyclictest5893-21apt-get11:15:133
17649991512,3cyclictest381rcuc/310:45:203
1764599152,6cyclictest0-21swapper/210:15:152
17643991512,2cyclictest18580-21latency_hist10:35:011
17643991512,2cyclictest18580-21latency_hist10:35:001
1764399151,13cyclictest0-21swapper/108:35:131
1763699152,7cyclictest6601-21munin-run12:25:000
1764999143,10cyclictest0-21swapper/310:40:203
17649991410,3cyclictest19837-21wc08:20:193
17649991410,3cyclictest11956-21tr08:05:153
17649991410,1cyclictest39-21ksoftirqd/312:20:233
1764599149,3cyclictest0-21swapper/207:10:192
1764599148,2cyclictest32-21ksoftirqd/207:50:142
17645991411,2cyclictest2591-21grep11:05:262
17645991410,3cyclictest20842-21grep09:30:182
1764399145,4cyclictest225-21systemd-journal11:20:001
1764399142,2cyclictest225-21systemd-journal07:15:001
17643991411,2cyclictest24754-21cron09:40:001
17643991411,2cyclictest20133-21seq11:45:001
1763699146,2cyclictest27127-21basename12:00:000
1763699142,6cyclictest23369-21awk10:45:010
17636991411,2cyclictest8014-21perf11:20:000
17636991411,2cyclictest28612-21cut10:55:140
17636991410,3cyclictest491-21grep09:55:170
17636991410,3cyclictest4243-21cron10:05:000
1764999139,3cyclictest30529-21grep09:50:173
1764999139,3cyclictest26711-21grep07:25:263
1764999138,4cyclictest26997-21hddtemp_smartct10:50:193
1764999133,2cyclictest225-21systemd-journal12:04:593
17649991310,2cyclictest696-21sh08:50:003
17649991310,2cyclictest25758-21cron08:34:593
17649991310,2cyclictest22682-21cut11:50:123
17649991310,2cyclictest21957-21tail09:30:273
1764599139,3cyclictest19121-21tr08:20:142
1764599139,2cyclictest5750-21tr10:05:202
1764599139,2cyclictest11800-21grep08:05:152
17645991310,2cyclictest5627-21munin-plugin-st11:14:592
17645991310,2cyclictest29557-21cron07:35:002
17645991310,2cyclictest28604-21tr12:00:222
17645991310,2cyclictest11419-21cron08:05:002
1764399139,3cyclictest23950-21head07:20:251
17643991310,2cyclictest9937-21grep09:05:251
1763699139,3cyclictest31407-21grep09:50:270
1763699139,3cyclictest30002-21grep12:05:150
1763699139,3cyclictest23944-21grep07:20:250
1763699139,3cyclictest13702-21sh12:40:010
1763699138,4cyclictest8527-21unixbench_multi10:10:260
1763699135,2cyclictest225-21systemd-journal09:10:010
1763699131,3cyclictest225-21systemd-journal09:44:590
17636991310,2cyclictest4057-21head08:55:200
17636991310,2cyclictest2193-21grep08:50:210
1763699130,7cyclictest20977-21awk08:24:590
1764999129,2cyclictest7606-21tail10:10:183
1764999129,2cyclictest6502-21cut09:00:193
1764999129,2cyclictest39-21ksoftirqd/307:50:003
1764999129,2cyclictest30509-21cut07:35:163
1764999129,2cyclictest29893-21cut12:05:163
1764999129,2cyclictest24310-21tail09:35:303
1764999129,2cyclictest21735-21wc07:15:263
1764999129,2cyclictest19425-21grep11:40:253
1764999129,2cyclictest1635-21head11:05:173
1764999129,2cyclictest1434-21grep09:55:273
1764999129,2cyclictest12332-21sed11:25:273
1764999129,2cyclictest10767-21wc08:00:263
1764999128,3cyclictest3108-21wc10:00:213
1764999128,3cyclictest25720-21grep11:55:193
1764999128,3cyclictest14068-21grep10:25:133
1764999127,4cyclictest39-21ksoftirqd/307:10:193
17649991210,1cyclictest14074-21apt-get08:10:143
1764999120,4cyclictest32207-21tr11:00:253
1764599129,2cyclictest32-21ksoftirqd/212:05:272
1764599129,2cyclictest3084-21basename08:55:012
1764599129,2cyclictest30566-21head07:35:182
1764599129,2cyclictest29085-21sed09:45:242
1764599129,2cyclictest2792-21grep12:15:152
1764599129,2cyclictest23538-21cat08:30:012
1764599129,2cyclictest23288-21grep11:50:162
1764599129,2cyclictest22963-21grep07:20:172
1764599129,2cyclictest19049-21cut10:35:172
1764599129,2cyclictest16664-21grep10:30:162
1764599129,2cyclictest16664-21grep10:30:152
1764599129,2cyclictest16087-21taskset09:20:182
1764599129,2cyclictest13844-21head09:15:182
1764599129,2cyclictest11482-21head09:10:162
1764599128,3cyclictest4255-21sh10:05:002
1764599126,2cyclictest32-21ksoftirqd/212:10:272
1764599125,3cyclictest225-21systemd-journal08:00:002
1764599122,2cyclictest141rcu_preempt11:55:212
17645991211,1cyclictest311rcuc/210:20:132
1764399129,2cyclictest9548-21wc11:20:221
1764399129,2cyclictest7710-21grep07:55:181
1764399129,2cyclictest767-21cat09:55:201
1764399129,2cyclictest3918-21taskset12:16:381
1764399129,2cyclictest27440-21grep10:50:231
1764399129,2cyclictest26670-21grep09:40:251
1764399129,2cyclictest26020-21wc11:55:211
1764399129,2cyclictest23902-21grep10:45:131
1764399129,2cyclictest16189-21grep09:20:181
1764399129,2cyclictest0-21swapper/107:40:111
1764399126,5cyclictest9302-21tr10:15:151
1764399121,10cyclictest0-21swapper/109:15:211
1764399120,10cyclictest0-21swapper/111:25:131
1763699129,2cyclictest675-21cron08:50:010
1763699129,2cyclictest28248-21head07:30:180
1763699129,2cyclictest20894-21idleruntime-cro10:40:010
1763699129,2cyclictest16164-21head11:35:180
1763699129,2cyclictest13620-21grep09:15:160
1763699129,2cyclictest10854-21grep12:30:260
1763699128,3cyclictest28150-21sed12:00:180
1763699124,7cyclictest15475-21apt-get09:20:150
1763699121,6cyclictest19979-21idleruntime-cro09:30:000
1763699121,5cyclictest32082-21cut11:00:210
1764999119,2cyclictest381rcuc/308:55:243
1764999119,1cyclictest29495-21chrt10:55:183
1764999118,2cyclictest7676-21sed07:55:183
1764999118,2cyclictest5285-21tail07:50:153
1764999118,2cyclictest4609-21cut11:10:203
1764999118,2cyclictest28150-21grep09:45:163
1764999118,2cyclictest27012-21cut08:35:173
1764999118,2cyclictest26695-21tail09:40:263
1764999118,2cyclictest21816-21sort11:45:273
1764999118,2cyclictest17369-21grep08:15:193
1764999118,2cyclictest17105-21grep11:35:273
1764999118,2cyclictest14333-21wc09:15:233
1764999118,2cyclictest1383-21tail12:10:283
1764999118,2cyclictest12637-21sort12:35:213
1764999118,2cyclictest11404-21sleep10:20:003
1764999118,2cyclictest11325-21cron12:35:003
1764999114,3cyclictest39-21ksoftirqd/309:09:523
1764999112,3cyclictest0-21swapper/307:30:153
1764599118,2cyclictest6600-21munin-plugin-st12:25:012
1764599118,2cyclictest32671-21grep11:00:292
1764599118,2cyclictest32461-21tr07:40:152
1764599118,2cyclictest28222-21taskset10:54:032
1764599118,2cyclictest2760-21diskstats07:45:162
1764599118,2cyclictest25784-21fw_packets07:25:162
1764599118,2cyclictest25040-21basename09:40:112
1764599118,2cyclictest24908-21cut08:30:552
1764599118,2cyclictest24215-21sort10:45:162
1764599118,2cyclictest23419-21wc10:45:002
1764599118,2cyclictest21731-21wc07:15:262
1764599118,1cyclictest32-21ksoftirqd/208:15:182
1764599117,3cyclictest32-21ksoftirqd/209:00:172
1764599117,3cyclictest31896-21cron09:55:002
1764599117,3cyclictest26524-21grep08:35:142
1764599117,3cyclictest11517-21cat12:35:002
1764599111,3cyclictest0-21swapper/209:55:192
1764599111,2cyclictest225-21systemd-journal12:40:012
1764599110,10cyclictest19217-21swap11:40:242
1764399119,1cyclictest32380-21grep12:10:161
1764399119,1cyclictest21947-21grep09:30:271
1764399119,1cyclictest21542-21grep08:25:151
1764399119,1cyclictest20986-21grep07:15:171
1764399119,1cyclictest18568-21grep09:25:171
1764399119,1cyclictest12397-21grep10:20:171
1764399118,2cyclictest8991-21cron10:15:001
1764399118,2cyclictest31784-21sed08:45:171
1764399118,2cyclictest31306-21cat09:50:271
1764399118,2cyclictest28128-21grep12:00:181
1764399118,2cyclictest2446-21grep07:45:151
1764399118,2cyclictest19946-21cat08:20:181
1764399118,2cyclictest18899-21cut11:40:191
1764399118,2cyclictest15778-21cat11:35:151
1764399118,2cyclictest14830-21grep08:10:181
1764399118,2cyclictest14297-21grep10:25:141
1764399116,2cyclictest29851-21cut12:05:151
1764399112,4cyclictest6639-21munin-run10:10:001
1764399110,10cyclictest1161-21grep11:05:141
1763699118,2cyclictest931-21head12:10:230
1763699118,2cyclictest8007-21ntp_states12:25:220
1763699118,2cyclictest6353-21fschecks_count09:00:170
1763699118,2cyclictest510-21grep07:40:160
1763699118,2cyclictest4740-21tr10:05:140
1763699118,2cyclictest4550-21apache_accesses07:50:110
1763699118,2cyclictest3792-21grep12:15:260
1763699118,2cyclictest31953-21ls07:40:000
1763699118,2cyclictest28322-21chrt08:39:480
1763699118,2cyclictest2646-21head11:05:260
1763699118,2cyclictest25755-21cron08:35:000
1763699118,2cyclictest22900-21grep08:25:260
1763699118,2cyclictest22863-21grep11:50:130
1763699118,2cyclictest21550-21tr07:15:220
1763699118,2cyclictest19489-21sed07:10:240
1763699118,2cyclictest16960-21grep10:30:170
1763699118,2cyclictest16960-21grep10:30:160
1763699118,2cyclictest14257-21grep11:30:250
1763699118,2cyclictest11963-21cat10:20:140
1763699118,2cyclictest10923-21sed10:15:270
1763699117,3cyclictest22382-21cron09:34:590
1764999108,1cyclictest232332sleep308:28:103
1764999108,1cyclictest22713-21chrt07:20:153
1764999108,1cyclictest20693-21chrt10:37:323
1764999108,1cyclictest174062sleep309:22:593
1764999108,1cyclictest134622sleep310:21:123
1764999108,1cyclictest126222sleep309:12:493
1764999107,2cyclictest6896-21basename12:25:153
1764999107,2cyclictest5194-21sort10:05:153
1764999107,2cyclictest2553-21cat08:50:263
1764999107,2cyclictest19273-21sort09:25:253
1764599108,1cyclictest7189-21wc10:10:162
1764599108,1cyclictest28999-21expr10:55:142
1764599108,1cyclictest15381-21cat11:35:122
1764599108,1cyclictest14778-21head08:10:182
1764599108,1cyclictest13621-21entropy11:30:192
1764599108,1cyclictest12490-21taskset11:26:362
1764599107,2cyclictest8832-21df11:20:162
1764599107,2cyclictest4410-21sed08:55:222
1764599107,2cyclictest31277-21expr08:45:142
1764599107,2cyclictest24718-21chrt09:39:372
1764599100,9cyclictest83152sleep209:05:112
1764399108,2cyclictest11152-21taskset12:32:491
1764399108,1cyclictest5382-21cat12:20:191
1764399108,1cyclictest29016-21expr09:45:241
1764399108,1cyclictest21347-21wget10:40:151
1764399107,2cyclictest30673-21cron11:00:001
1764399107,2cyclictest30246-21grep08:40:261
1764399107,2cyclictest29357-21chrt07:32:271
1764399107,2cyclictest26611-21cat07:25:251
1764399107,2cyclictest22820-21grep11:50:131
1764399107,2cyclictest12804-21idleruntime-cro09:15:001
1764399107,2cyclictest11831-21expr12:35:161
1764399105,2cyclictest225-21systemd-journal08:35:001
1764399104,2cyclictest225-21systemd-journal12:26:191
1764399103,5cyclictest18745-21sshd10:35:021
1764399100,9cyclictest0-21swapper/108:05:171
1763699108,1cyclictest9094-21cat11:20:180
1763699108,1cyclictest3014-21basename07:45:180
1763699108,1cyclictest27877-21expr09:45:140
1763699108,1cyclictest20634-21expr11:45:160
1763699108,1cyclictest18407-21taskset08:18:100
1763699108,1cyclictest14339-21grep08:10:150
1763699107,2cyclictest7573-21chrt07:55:170
1763699107,2cyclictest26510-21cat07:25:250
1763699107,2cyclictest19924-21sleep11:44:590
1763699107,2cyclictest12490-21taskset09:11:060
1763699107,2cyclictest10950-21cut11:25:140
1763699107,2cyclictest10771-21cat08:00:260
176499997,1cyclictest8421-21cut11:20:143
176499997,1cyclictest17467-21wc10:30:213
176499997,1cyclictest17467-21wc10:30:213
176499997,1cyclictest148812sleep311:31:493
176459997,1cyclictest7153-21cat12:25:162
176459997,1cyclictest17856-21dpkg09:25:132
176459997,1cyclictest14181-21wc10:25:142
176459995,3cyclictest1142-21runrttasks08:40:192
176439998,1cyclictest3565-21expr08:55:181
176439998,1cyclictest14198-21ntpq11:30:241
176439998,1cyclictest0-21swapper/110:00:241
176439997,1cyclictest5742-21basename09:00:141
176439997,1cyclictest32210-21grep11:00:251
176439997,1cyclictest31193-21which07:35:241
176369997,1cyclictest26313-21grep10:50:160
176369997,1cyclictest11668-21cat08:05:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional