You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 04:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Dec 14, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30031210263,34sleep00-21swapper/019:09:340
2973229565,9sleep30-21swapper/319:05:453
2991229265,9sleep20-21swapper/219:08:042
3006528467,13sleep10-21swapper/119:09:591
30161993229,2cyclictest311rcuc/200:08:572
3016199260,1cyclictest0-21swapper/220:05:272
3016199210,20cyclictest0-21swapper/200:00:122
3015699210,10cyclictest0-21swapper/100:35:151
3016199200,19cyclictest0-21swapper/223:30:142
3016399182,1cyclictest0-21swapper/323:00:143
3015399180,17cyclictest0-21swapper/019:55:150
3016199172,1cyclictest141rcu_preempt20:05:002
30161991713,3cyclictest32-21ksoftirqd/223:20:192
3015399173,7cyclictest7177-21munin-run21:45:010
3016399155,7cyclictest22931-21apt-get23:25:153
3016399154,9cyclictest20821-21apt-get21:05:153
30163991511,3cyclictest25091-21cut20:05:173
30163991510,3cyclictest2810-21grep19:20:143
30161991513,1cyclictest0-21swapper/200:20:142
3015699154,7cyclictest0-21swapper/100:05:141
3015699154,6cyclictest225-21systemd-journal00:22:101
3015399150,8cyclictest13166-21perf23:05:000
30163991410,3cyclictest22272-21grep20:00:143
3016199149,4cyclictest0-21swapper/220:34:592
30161991410,4cyclictest311rcuc/222:40:252
30156991410,3cyclictest30350-21wc22:30:261
3016399139,3cyclictest7417-21tr00:00:133
3016399139,3cyclictest27201-21grep21:15:283
3016399139,3cyclictest11484-21sed21:50:263
3016399139,2cyclictest6504-21tr20:35:153
3016399139,2cyclictest30379-21grep21:25:143
3016399139,2cyclictest30322-21wc22:30:273
30163991310,2cyclictest6682-21wc19:25:253
30163991310,2cyclictest6235-21tr22:50:133
30163991310,2cyclictest27968-21wc22:25:243
30163991310,2cyclictest26806-21wc23:30:243
30163991310,2cyclictest26300-21grep00:35:273
30163991310,2cyclictest25823-21grep22:20:273
30163991310,2cyclictest23996-21wc22:19:593
30163991310,2cyclictest17886-21sh23:15:013
30163991310,2cyclictest15640-21sed00:15:183
30163991310,2cyclictest14195-21grep19:40:263
3016199139,3cyclictest29505-21wc21:20:252
3016199139,3cyclictest28493-21grep20:10:252
30161991310,2cyclictest631-21grep19:15:282
30161991310,2cyclictest2844-21grep19:20:142
30161991310,2cyclictest20076-21cut22:10:152
30161991310,2cyclictest18701-21grep22:05:282
30161991310,2cyclictest17662-21grep20:55:252
30161991310,2cyclictest15073-21grep20:50:272
30161991310,2cyclictest11762-21tail23:00:172
30161991310,2cyclictest11391-21wc19:35:242
30161991310,2cyclictest11391-21wc19:35:232
3015699139,3cyclictest27975-21basename21:20:141
30156991310,2cyclictest9147-21tail21:45:251
30156991310,2cyclictest23209-21tr23:25:151
30156991310,2cyclictest15546-21cat23:10:001
3015699130,12cyclictest20260-21cron23:20:011
3015399139,3cyclictest4876-21cat19:25:000
3015399139,3cyclictest3622-21sh20:30:000
30153991310,2cyclictest28728-21cat23:35:200
30153991310,2cyclictest23650-21sed00:30:270
30153991310,2cyclictest18504-21tr19:50:220
3015399130,8cyclictest9889-21perf19:35:000
3016399129,2cyclictest2898-21grep21:35:143
3016399129,2cyclictest27033-21tr20:10:133
3016399129,2cyclictest18885-21head23:15:143
3016399129,2cyclictest17755-21grep20:55:253
3016399129,2cyclictest1736-21sed23:45:263
3016399129,2cyclictest12809-21head20:45:293
3016399129,2cyclictest11252-21grep00:05:283
3016399128,2cyclictest9939-21grep20:40:193
3016399126,5cyclictest1252-21idleruntime-cro20:25:003
3016399121,8cyclictest0-21swapper/300:25:013
3016199129,2cyclictest6171-21grep21:40:202
3016199129,2cyclictest26239-21tail00:35:282
3016199129,2cyclictest24117-21munin-run00:34:592
3016199129,2cyclictest15587-21wc23:10:002
3016199129,2cyclictest11131-21wc21:50:222
3016199129,2cyclictest10530-21cron23:00:002
3016199128,3cyclictest9872-21dump-pmu-power19:35:002
3016199128,3cyclictest4316-21grep21:35:252
3016199128,3cyclictest20882-21wc00:25:212
3016199122,2cyclictest225-21systemd-journal23:41:192
3016199121,2cyclictest32-21ksoftirqd/221:16:562
3016199120,3cyclictest0-21swapper/222:45:192
3015699129,2cyclictest9457-21cat22:55:191
3015699129,2cyclictest7224-21grep19:30:001
3015699129,2cyclictest32326-21cat23:45:011
3015699129,2cyclictest32047-21tail19:10:221
3015699129,2cyclictest24269-21wc00:35:001
3015699129,2cyclictest2275-21head20:25:161
3015699129,2cyclictest21861-21tr22:15:141
3015699129,2cyclictest18226-21ls22:05:231
3015699129,2cyclictest12239-21grep20:45:231
3015699128,2cyclictest19624-21tr22:10:131
3015699126,2cyclictest225-21systemd-journal20:19:591
3015699121,8cyclictest32197-21kworker/1:219:30:131
3015699120,11cyclictest1799-21df_abs22:40:181
3015699120,0cyclictest0-21swapper/121:35:111
3015399129,2cyclictest902-21grep21:30:140
3015399129,2cyclictest902-21grep21:30:140
3015399129,2cyclictest31149-21cut19:10:140
3015399129,2cyclictest27493-21head20:10:160
3015399129,2cyclictest24911-21head22:20:190
3015399129,2cyclictest21571-21sed21:05:190
3015399129,2cyclictest18795-21wc00:20:250
3015399129,2cyclictest17247-21tr22:05:150
3015399129,2cyclictest17136-21wc00:20:000
3015399129,2cyclictest14084-21grep23:05:170
3015399129,2cyclictest13891-21sed21:55:250
3015399129,2cyclictest13282-21sed19:40:160
3015399125,6cyclictest25055-21perf23:30:000
3016399118,2cyclictest9475-21taskset21:48:303
3016399118,2cyclictest7038-21chrt21:43:263
3016399118,2cyclictest31740-21systemctl19:10:183
3016399118,2cyclictest3164-21taskset22:42:383
3016399118,2cyclictest29821-21sendmail-msp23:40:003
3016399118,2cyclictest29121-21chrt20:15:073
3016399118,2cyclictest20827-21wc22:10:233
3016399118,2cyclictest19073-21chrt22:09:363
3016399118,2cyclictest16752-21taskset19:46:543
3016399118,2cyclictest1618-21fgrep21:30:233
3016399118,2cyclictest1618-21fgrep21:30:223
3016399118,2cyclictest15060-21tail23:05:263
3016399117,2cyclictest2465-21grep23:50:133
3016399113,3cyclictest225-21systemd-journal20:34:593
3016399112,6cyclictest21742-21cron20:00:013
3016399110,10cyclictest21402-21taskset00:25:553
3016199118,3cyclictest32-21ksoftirqd/200:13:562
3016199118,2cyclictest9419-21grep20:40:152
3016199118,2cyclictest9310-21chrt21:46:222
3016199118,2cyclictest7933-21grep20:35:262
3016199118,2cyclictest3344-21chrt20:26:322
3016199118,2cyclictest32-21ksoftirqd/222:39:592
3016199118,2cyclictest30414-21cut19:10:102
3016199118,2cyclictest29497-21sort22:30:172
3016199118,2cyclictest28061-21expr23:35:152
3016199118,2cyclictest25389-21grep22:20:232
3016199118,2cyclictest24597-21grep23:25:282
3016199118,2cyclictest22430-21sed21:05:282
3016199118,2cyclictest1773-21taskset23:45:312
3016199118,2cyclictest16568-21sed23:10:182
3016199118,2cyclictest16290-21grep22:00:252
3016199118,2cyclictest14233-21grep19:40:252
3016199118,2cyclictest13186-21cron20:49:592
3016199117,3cyclictest32-21ksoftirqd/221:25:122
3016199117,3cyclictest19816-21tail23:15:242
3016199115,2cyclictest225-21systemd-journal20:00:012
3016199110,10cyclictest5925-21ntp_states23:55:222
3016199110,0cyclictest0-21swapper/200:15:142
3015699119,2cyclictest25-21ksoftirqd/120:35:131
3015699119,1cyclictest19170-21grep21:00:171
3015699118,2cyclictest6871-21cron00:00:011
3015699118,2cyclictest5960-21taskset22:50:071
3015699118,2cyclictest32400-21tr22:35:211
3015699118,2cyclictest27398-21taskset21:16:381
3015699118,2cyclictest27358-21df20:10:151
3015699118,2cyclictest27348-21basename22:25:171
3015699118,2cyclictest26395-21grep23:30:191
3015699118,2cyclictest24570-21expr22:20:171
3015699118,2cyclictest23866-21grep21:10:161
3015699118,2cyclictest21359-21grep19:55:251
3015699118,2cyclictest19466-21cat19:55:001
3015699118,2cyclictest16332-21grep23:10:161
3015699118,2cyclictest14908-21expr22:00:141
3015699118,2cyclictest1150-21tr23:45:201
3015699117,3cyclictest4424-21sed19:20:251
3015699117,3cyclictest168912sleep100:19:111
3015699114,3cyclictest225-21systemd-journal19:40:011
3015699114,3cyclictest225-21systemd-journal19:40:001
3015699110,9cyclictest0-21swapper/120:05:141
3015399118,2cyclictest7342-21wc19:30:010
3015399118,2cyclictest6669-21ls20:35:140
3015399118,2cyclictest4742-21taskset21:39:410
3015399118,2cyclictest4179-21tr20:30:170
3015399118,2cyclictest3354-21sh22:45:000
3015399118,2cyclictest32288-21grep20:20:160
3015399118,2cyclictest29722-21chrt21:20:410
3015399118,2cyclictest27288-21head22:25:160
3015399118,2cyclictest26512-21ls23:30:210
3015399118,2cyclictest25837-21grep21:15:150
3015399118,2cyclictest25151-21grep20:05:180
3015399118,2cyclictest23041-21cat20:00:190
3015399118,2cyclictest20001-21chrt23:16:480
3015399117,3cyclictest21607-21cat23:20:180
3015399117,3cyclictest11645-21ls19:35:270
3015399117,3cyclictest11645-21ls19:35:260
3015399115,5cyclictest11819-21apache200:35:240
3015399113,4cyclictest225-21systemd-journal00:09:040
3016399108,1cyclictest8467-21grep22:55:133
3016399108,1cyclictest3010-21cat20:25:233
3016399108,1cyclictest29125-21sh21:20:213
3016399108,1cyclictest24985-21taskset21:11:063
3016399108,1cyclictest21956-21taskset00:30:043
3016399108,1cyclictest18241-21sed19:50:193
3016399108,1cyclictest140812sleep300:11:273
3016399108,1cyclictest13517-21aten2_r7power_e20:50:153
3016399107,2cyclictest3852-21tr22:45:143
3016399107,2cyclictest301-21bc19:15:253
3016399107,2cyclictest19761-21grep21:00:233
3016399107,2cyclictest16353-21head22:00:263
3016399107,2cyclictest13010-21cut21:55:153
3016399107,2cyclictest10130-21apache_accesses19:35:133
3016399107,2cyclictest10130-21apache_accesses19:35:133
3016399106,2cyclictest31504-21tr22:35:143
3016399104,2cyclictest516-21gdbus23:20:213
3016399100,9cyclictest5227-21expr23:55:153
3016199108,1cyclictest710-21expr21:30:142
3016199108,1cyclictest710-21expr21:30:132
3016199108,1cyclictest69462sleep219:26:382
3016199108,1cyclictest29571-21cat20:15:152
3016199108,1cyclictest15823-21grep19:45:192
3016199108,1cyclictest13179-21cut21:55:172
3016199107,2cyclictest22303-21grep22:15:152
3016199107,2cyclictest19058-21cut21:00:162
3016199107,2cyclictest18502-21grep19:50:222
3016199107,2cyclictest1253-21dump-pmu-power20:24:592
3016199105,3cyclictest6118-21grep22:50:122
3015699108,1cyclictest9973-21grep21:50:141
3015699108,1cyclictest7313-21wc00:00:131
3015699108,1cyclictest5439-21expr21:40:151
3015699108,1cyclictest38612sleep120:30:121
3015699108,1cyclictest2251-21taskset21:33:131
3015699108,1cyclictest2251-21taskset21:33:121
3015699108,1cyclictest22461-21expr20:00:141
3015699108,1cyclictest21685-21ntpq23:20:191
3015699108,1cyclictest12682-21ls21:55:131
3015699108,1cyclictest11290-21cut23:00:151
3015699107,2cyclictest9205-21sed20:40:131
3015699107,2cyclictest4240-21tune2fs22:45:151
3015699107,2cyclictest28390-21tail23:35:171
3015699107,2cyclictest20012-21ls00:25:151
3015699107,2cyclictest15855-21wc19:45:201
3015699107,2cyclictest12762-21tr19:40:141
3015699106,3cyclictest13517-21wc00:10:221
3015699100,9cyclictest31035-21grep21:25:171
3015399108,1cyclictest7736-21df_inode00:00:140
3015399108,1cyclictest7409-21cat21:45:110
3015399108,1cyclictest4776-21apt23:55:140
3015399108,1cyclictest443-21aten2_r7power_v19:15:280
3015399108,1cyclictest30901-21taskset22:33:440
3015399108,1cyclictest2820-21expr23:50:130
3015399108,1cyclictest25195-21taskset21:13:460
3015399108,1cyclictest19873-21cut00:25:140
3015399108,1cyclictest17849-21taskset23:14:370
3015399108,1cyclictest15733-21sed22:00:190
3015399108,1cyclictest14947-21cat20:50:250
3015399108,1cyclictest12257-21apache_accesses00:10:120
3015399108,1cyclictest10988-21ntp_kernel_err21:50:190
3015399108,1cyclictest10473-21taskset20:40:390
3015399107,2cyclictest7383-21sensors22:50:220
3015399107,2cyclictest595-21chrt22:35:330
3015399107,2cyclictest31968-21grep21:25:260
3015399107,2cyclictest29931-21tail20:15:170
3015399107,2cyclictest22874-21ntp_kernel_err22:15:200
3015399107,2cyclictest20157-21sed21:00:260
3015399107,2cyclictest19640-21grep22:10:140
3015399107,2cyclictest10579-21lockfile-create23:00:000
3015399106,2cyclictest15003-21grep19:45:150
3015399104,3cyclictest225-21systemd-journal20:49:590
3015399100,9cyclictest31230-21ntp_states23:40:200
301639997,1cyclictest77012sleep319:30:123
301639997,1cyclictest319452sleep323:42:113
301619997,1cyclictest24660-21sed21:10:252
301619997,1cyclictest23962sleep223:50:112
301619994,4cyclictest311rcuc/222:25:202
301569998,1cyclictest17188-21ntpq20:55:201
301569997,1cyclictest31516-21date20:20:001
301569997,1cyclictest2467-21tr23:50:121
301569997,1cyclictest20781-21chrt21:05:111
301569997,1cyclictest152242sleep120:50:441
301569994,2cyclictest225-21systemd-journal19:16:101
301539997,1cyclictest178622sleep020:56:230
301539996,2cyclictest3875-21grep22:45:130
301539996,2cyclictest32715-21tr23:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional