You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 15:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sun Dec 28, 2025 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2252829171,15sleep10-21swapper/107:07:271
2241528744,39sleep00-21swapper/007:05:570
2269628262,9sleep30-21swapper/307:09:383
2059528168,9sleep20-21swapper/207:05:072
190802660,1sleep319081-21grep11:30:173
2281499215,6cyclictest10962-21idleruntime-cro11:15:000
2281799201,2cyclictest0-21swapper/110:50:191
2282099191,2cyclictest23225-21apache_accesses09:25:112
22817991715,1cyclictest0-21swapper/110:20:141
22814991710,7cyclictest0-21swapper/011:30:140
2282099150,15cyclictest0-21swapper/208:15:142
2281499155,4cyclictest11012-21sendmail09:00:000
22820991410,3cyclictest30148-21grep11:55:012
2281799143,8cyclictest6236-21idleruntime-cro08:49:591
2281799143,7cyclictest225-21systemd-journal10:31:281
2281499149,3cyclictest29164-21cat10:44:590
2281499143,9cyclictest9503-21cron12:19:590
22814991410,2cyclictest26301-21grep10:35:300
2282499139,3cyclictest19128-21tail09:15:173
2282499139,3cyclictest12202-21sh07:55:003
2282499139,3cyclictest10437-21cat08:55:243
22824991310,2cyclictest870-21sort12:00:193
22824991310,2cyclictest7110-21sort11:05:183
22824991310,2cyclictest26287-21tail11:45:143
22824991310,2cyclictest24144-21wc11:40:193
22824991310,2cyclictest23790-21cat10:30:263
22824991310,2cyclictest18368-21wc10:20:203
22824991310,2cyclictest17724-21grep09:10:253
22824991310,2cyclictest15741-21grep11:25:003
22824991310,2cyclictest11757-21grep10:05:273
2282099139,3cyclictest5807-21tail12:10:172
2282099137,2cyclictest225-21systemd-journal10:10:012
2282099136,3cyclictest225-21systemd-journal10:19:592
2282099132,4cyclictest225-21systemd-journal10:25:002
22820991310,2cyclictest29445-21wc09:35:242
22820991310,2cyclictest29445-21wc09:35:232
22820991310,2cyclictest24905-21grep07:10:272
22820991310,2cyclictest20532-21munin-run11:35:002
22817991310,2cyclictest6747-21sed12:10:271
22817991310,2cyclictest27743-21cron07:19:591
22817991310,2cyclictest15478-21grep12:30:181
22817991310,2cyclictest15245-21tr10:15:141
2281499139,3cyclictest16617-21sed10:15:260
2281499133,3cyclictest225-21systemd-journal08:15:010
2281499131,6cyclictest3875-21munin-run08:45:000
22814991310,2cyclictest7948-21tr07:45:160
22814991310,2cyclictest29613-21sed11:50:250
22814991310,2cyclictest28319-21grep07:20:390
22814991310,2cyclictest21275-21grep11:35:150
22814991310,2cyclictest19626-21sort10:25:000
2282499129,2cyclictest8868-21grep11:10:153
2282499129,2cyclictest6848-21grep07:40:263
2282499129,2cyclictest6848-21grep07:40:253
2282499129,2cyclictest6496-21ls09:55:223
2282499129,2cyclictest5027-21turbostat09:55:013
2282499129,2cyclictest4393-21tail07:35:293
2282499129,2cyclictest30559-21sed10:45:233
2282499129,2cyclictest27258-21grep10:40:153
2282499129,2cyclictest27090-21cut08:25:143
2282499129,2cyclictest26775-21apt08:25:003
2282499129,2cyclictest26552-21grep09:30:183
2282499129,2cyclictest15007-21tr12:30:163
2282499129,2cyclictest14457-21wc09:05:193
2282499129,2cyclictest11151-21tr12:20:213
2282499128,3cyclictest8458-21head10:00:193
2282499125,3cyclictest225-21systemd-journal08:49:593
2282099129,2cyclictest9328-21grep10:00:282
2282099129,2cyclictest8498-21head07:45:182
2282099129,2cyclictest3930-21wc08:45:002
2282099129,2cyclictest31147-21head07:25:162
2282099129,2cyclictest27313-21grep08:25:152
2282099129,2cyclictest25371-21grep10:35:202
2282099129,2cyclictest24171-21grep11:40:192
2282099129,2cyclictest22930-21head10:30:172
2282099129,2cyclictest1986-21sed07:30:252
2282099129,2cyclictest14339-21grep11:20:162
2282099129,2cyclictest14050-21grep10:10:282
2282099129,2cyclictest10974-21cron08:59:592
2282099129,2cyclictest10688-21cut12:20:172
2281799129,2cyclictest9529-21sleep12:19:591
2281799129,2cyclictest6386-21grep09:55:211
2281799129,2cyclictest23461-21grep11:40:151
2281799129,2cyclictest19335-21cron08:10:001
2281799129,2cyclictest15623-21grep08:00:201
2281799129,2cyclictest12174-21awk10:10:011
2281799129,2cyclictest1089-21head07:30:161
2281799128,3cyclictest31528-21perf08:34:591
22817991210,1cyclictest16692-21grep09:10:161
2281499129,2cyclictest8560-21cron11:10:000
2281499129,2cyclictest5789-21grep11:00:290
2281499129,2cyclictest31985-21tr10:50:170
2281499129,2cyclictest31349-21head09:40:180
2281499129,2cyclictest26323-21grep08:20:260
2281499129,2cyclictest2406-21perf12:05:000
2281499129,2cyclictest14268-21sort09:05:180
2281499129,2cyclictest12526-21tr12:25:170
2281499129,2cyclictest12037-21sed09:00:170
2281499129,2cyclictest10314-21grep10:05:150
2281499128,3cyclictest26355-21grep07:15:180
2281499128,2cyclictest10149-21cut12:20:150
2281499120,9cyclictest0-21swapper/008:25:140
2281499120,11cyclictest0-21swapper/009:45:120
2282499119,1cyclictest22418-21grep11:35:263
2282499118,2cyclictest7235-21grep08:50:173
2282499118,2cyclictest5312-21tr12:10:153
2282499118,2cyclictest32458-21taskset07:29:273
2282499118,2cyclictest32003-21tr08:35:163
2282499118,2cyclictest31506-21cron08:34:593
2282499118,2cyclictest2647-21munin-run09:50:003
2282499118,2cyclictest2468-21head08:40:163
2282499118,2cyclictest23343-21cut09:25:143
2282499118,2cyclictest23013-21cut08:15:183
2282499118,2cyclictest21948-21sed09:20:213
2282499118,2cyclictest20090-21tr10:25:123
2282499118,2cyclictest17602-21expr08:05:163
2282499118,2cyclictest17048-21date08:05:013
2282499118,2cyclictest16525-21grep11:25:163
2282499118,2cyclictest15757-21sed10:15:183
2282499118,2cyclictest13426-21wc07:55:213
2282499118,2cyclictest13246-21if_enp2s012:25:193
2282499118,2cyclictest11225-21chrt09:00:043
2282499118,2cyclictest1020-21cut07:30:163
2282499117,3cyclictest32021-21sed11:55:263
2282499110,10cyclictest32025-21grep10:50:163
2282499110,10cyclictest21095-21cat08:10:253
2282099118,2cyclictest8582-21taskset08:54:522
2282099118,2cyclictest545-21tr12:00:172
2282099118,2cyclictest503-21apache_accesses09:45:102
2282099118,2cyclictest4836-21tail11:00:202
2282099118,2cyclictest4525-21taskset12:07:062
2282099118,2cyclictest4362-21tr08:45:142
2282099118,2cyclictest31995-21grep10:50:162
2282099118,2cyclictest30711-21grep10:45:242
2282099118,2cyclictest28749-21taskset10:40:382
2282099118,2cyclictest2764-21basename07:35:162
2282099118,2cyclictest27434-21sed09:30:272
2282099118,2cyclictest20462-21sort10:25:142
2282099118,2cyclictest19275-21basename12:40:002
2282099118,2cyclictest17574-21expr08:05:162
2282099118,2cyclictest16765-21expr11:25:182
2282099118,2cyclictest15547-21expr12:30:182
2282099118,2cyclictest11827-21fschecks_count11:15:172
2282099115,2cyclictest225-21systemd-journal09:25:002
2282099113,5cyclictest12523-21grep09:00:232
2282099110,10cyclictest14466-21chrt07:58:492
2281799119,1cyclictest4733-21grep11:00:191
2281799119,1cyclictest32125-21head07:25:261
2281799119,1cyclictest30732-21wc09:40:151
2281799119,1cyclictest13307-21cron11:20:011
2281799119,1cyclictest1011-21df09:45:141
2281799118,2cyclictest7690-21ls08:50:221
2281799118,2cyclictest29148-21cat09:35:201
2281799118,2cyclictest29148-21cat09:35:201
2281799118,2cyclictest28727-21grep08:25:281
2281799118,2cyclictest25117-21taskset09:27:211
2281799118,2cyclictest24365-21cron08:20:011
2281799118,2cyclictest21777-21sh08:15:011
2281799118,2cyclictest18405-21grep11:30:151
2281799118,2cyclictest16294-21cat11:25:161
2281799118,2cyclictest15303-21grep11:20:251
2281799118,2cyclictest14186-21sed07:55:271
2281799118,2cyclictest14041-21grep10:10:291
2281799118,2cyclictest11075-21cat09:00:001
2281799118,1cyclictest3033-21grep09:50:141
2281799118,1cyclictest2145-21expr08:40:151
2281799115,2cyclictest0-21swapper/109:30:151
2281799114,4cyclictest273-21systemd-udevd07:51:031
2281799111,8cyclictest8839-21apt-get11:10:141
22817991110,1cyclictest0-21swapper/109:15:131
2281799110,10cyclictest24867-21grep10:35:181
2281499119,1cyclictest13325-21if_enp2s007:55:190
2281499118,2cyclictest7234-21grep08:50:170
2281499118,2cyclictest3837-21cat07:35:220
2281499118,2cyclictest3229-21wc10:55:250
2281499118,2cyclictest30726-21grep08:30:280
2281499118,2cyclictest30594-21cut11:55:150
2281499118,2cyclictest29324-21taskset10:45:070
2281499118,2cyclictest28029-21wc09:35:010
2281499118,2cyclictest23352-21tail08:15:210
2281499118,2cyclictest21062-21grep10:25:200
2281499118,2cyclictest18323-21grep08:05:210
2281499118,2cyclictest17987-21chrt11:28:550
2281499118,2cyclictest16972-21cat12:34:590
2281499118,2cyclictest15296-21sed11:20:250
2281499114,6cyclictest18148-21perf09:14:590
2282499108,1cyclictest323712sleep309:41:033
2282499108,1cyclictest30063-21basename11:55:013
2282499108,1cyclictest29927-21chrt07:22:243
2282499108,1cyclictest26611-21chrt10:39:003
2282499108,1cyclictest2506-21sort10:55:183
2282499108,1cyclictest24575-21grep07:10:233
2282499108,1cyclictest191652sleep312:39:133
2282499107,2cyclictest9059-21grep12:15:283
2282499107,2cyclictest4304-21grep12:05:273
2282499107,2cyclictest29408-21sed09:35:243
2282499107,2cyclictest29408-21sed09:35:233
2282499107,2cyclictest14614-21cat10:14:593
2282499107,2cyclictest12818-21grep11:15:263
2282499106,3cyclictest5700-21sed11:00:293
2282499100,9cyclictest7939-21sort07:45:163
2282499100,9cyclictest27174-21wc07:15:263
2282099108,1cyclictest8956-21cut11:10:162
2282099108,1cyclictest5299-21taskset09:55:152
2282099108,1cyclictest32397-21fschecks_count08:35:182
2282099108,1cyclictest31388-21taskset08:33:322
2282099108,1cyclictest30645-21cut09:40:152
2282099108,1cyclictest30332-21hostname11:55:142
2282099108,1cyclictest25600-21cut08:20:192
2282099108,1cyclictest13990-21grep12:25:282
2282099107,2cyclictest8844-21wc12:15:272
2282099107,2cyclictest6528-21wget11:05:152
2282099107,2cyclictest29531-21cat07:20:502
2282099107,2cyclictest2402-21grep10:55:162
2282099107,2cyclictest20790-21chrt11:35:132
2282099107,2cyclictest19603-21apache_accesses08:10:122
2282099107,2cyclictest18619-21tr09:15:142
2282099107,2cyclictest16330-21wc09:10:142
2282099107,2cyclictest10402-21ls07:50:142
2282099106,3cyclictest14189-21df09:05:172
2282099105,2cyclictest452-21systemd-logind09:50:222
2282099104,4cyclictest26278-21fschecks_count07:15:172
2282099100,9cyclictest71032sleep207:43:252
2282099100,9cyclictest71032sleep207:43:252
2282099100,9cyclictest14829-21chrt08:00:132
2281799108,2cyclictest22575-21chrt11:36:041
2281799108,1cyclictest8131-21expr07:45:161
2281799108,1cyclictest501-21tr12:00:171
2281799108,1cyclictest3201-21df12:05:161
2281799108,1cyclictest31805-21cut08:35:151
2281799108,1cyclictest30034-21sh11:55:001
2281799108,1cyclictest28392-21cat07:20:391
2281799108,1cyclictest26324-21sed11:45:141
2281799107,2cyclictest9800-21perf10:05:001
2281799107,2cyclictest7006-21df11:05:171
2281799107,2cyclictest32547-21cat12:00:011
2281799107,2cyclictest29706-21cat10:45:171
2281799107,2cyclictest27193-21cat10:40:151
2281799107,2cyclictest1915-21cut10:55:141
2281799107,2cyclictest14792-21ntpq09:05:221
2281799107,2cyclictest11495-21grep09:00:151
2281799101,4cyclictest0-21swapper/112:25:161
2281799100,9cyclictest3044-21cat07:35:171
2281499108,1cyclictest94292sleep010:00:350
2281499108,1cyclictest4281-21tr09:50:220
2281499108,1cyclictest30557-21wc07:25:140
2281499108,1cyclictest25583-21cut11:45:120
2281499108,1cyclictest2136-21taskset07:30:480
2281499108,1cyclictest17934-21hostname12:35:210
2281499107,2cyclictest5828-21df09:55:160
2281499107,2cyclictest4308-21grep12:05:260
2281499107,2cyclictest29576-21sensors_fan09:35:250
2281499107,2cyclictest29576-21sensors_fan09:35:240
2281499107,2cyclictest22978-21tail10:30:160
2281499107,2cyclictest21094-21expr09:20:160
2281499107,2cyclictest20576-21sh09:20:000
2281499107,2cyclictest16753-21taskset08:02:250
2281499107,2cyclictest12902-21grep11:15:270
2281499107,2cyclictest12765-21expr10:10:170
2281499107,2cyclictest10292-21cat07:50:150
2281499106,3cyclictest23506-21tr09:25:140
2281499104,4cyclictest0-21swapper/008:35:140
2281499101,8cyclictest5031-21apache_volume12:10:140
2281499100,9cyclictest5828-21perl07:40:170
2281499100,9cyclictest5828-21perl07:40:160
2281499100,9cyclictest23210-21cut11:40:140
228179997,1cyclictest25355-21cron07:15:001
228179997,1cyclictest21196-21expr09:20:161
228179997,1cyclictest19981-21egrep10:25:121
228179997,1cyclictest17096-21seq12:35:101
228179997,1cyclictest10825-21idleruntime12:20:191
228149996,2cyclictest5903-21chrt08:45:540
228149996,2cyclictest25067-21chrt07:11:190
228209986,1cyclictest25911-21expr11:45:122
228179987,1cyclictest5133-21apache_volume07:40:121
228179987,1cyclictest5133-21apache_volume07:40:111
228179987,1cyclictest24965-21cat08:20:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional