You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 17:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Fri Jul 24, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5161210464,35sleep00-21swapper/007:06:350
526929878,15sleep10-21swapper/107:07:591
329928672,9sleep30-21swapper/307:05:123
510928371,8sleep20-21swapper/207:05:552
282462690,0sleep30-21swapper/311:23:223
286772660,1sleep328681-21grep10:25:193
5508992826,2cyclictest94-21kswapd009:35:030
550899270,24cyclictest0-21swapper/010:46:510
551799260,25cyclictest0-21swapper/310:04:003
551799244,2cyclictest20283-21ssh09:41:323
5508992411,12cyclictest23479-21sshd11:40:050
5517992322,1cyclictest0-21swapper/310:49:453
551799230,23cyclictest0-21swapper/311:17:483
550899234,10cyclictest0-21swapper/010:10:130
5517992120,1cyclictest0-21swapper/307:29:363
5517992120,1cyclictest0-21swapper/307:29:363
5515992119,1cyclictest0-21swapper/210:36:122
551199210,17cyclictest0-21swapper/109:23:161
551199210,17cyclictest0-21swapper/109:23:151
551799200,1cyclictest0-21swapper/309:38:143
551599206,13cyclictest0-21swapper/207:35:172
551199202,1cyclictest6555-21kworker/1:011:57:021
551199201,3cyclictest24443-21fw_packets11:40:161
550899200,0cyclictest0-21swapper/011:48:110
5515991917,1cyclictest0-21swapper/211:35:322
5515991917,1cyclictest0-21swapper/209:20:122
5515991917,1cyclictest0-21swapper/209:20:122
5515991916,2cyclictest32-21ksoftirqd/208:10:182
551199195,13cyclictest122402sleep109:56:001
551199190,4cyclictest0-21swapper/112:20:141
551199190,18cyclictest0-21swapper/112:39:491
550899192,16cyclictest30948-21apt-get11:25:140
550899190,0cyclictest0-21swapper/009:18:220
551599185,1cyclictest141rcu_preempt09:10:432
551199180,1cyclictest0-21swapper/109:53:411
5515991716,1cyclictest0-21swapper/209:25:322
5515991715,1cyclictest5655-21sh09:52:222
551599170,16cyclictest0-21swapper/209:39:532
551799167,7cyclictest1367-21pmu-power09:30:213
551799161,2cyclictest0-21swapper/310:20:033
5515991615,1cyclictest0-21swapper/212:12:522
551599160,14cyclictest0-21swapper/210:02:212
551199162,14cyclictest30732-21diskmemload12:32:031
551199162,14cyclictest30732-21diskmemload12:32:031
551199160,4cyclictest0-21swapper/110:15:181
551199160,16cyclictest0-21swapper/110:52:401
551199160,16cyclictest0-21swapper/110:52:401
5508991610,4cyclictest1932-21cron10:10:000
550899160,1cyclictest0-21swapper/012:21:370
550899160,1cyclictest0-21swapper/010:58:530
5517991514,1cyclictest0-21swapper/310:55:073
5517991511,3cyclictest23524-21ls09:25:003
5517991511,3cyclictest23524-21ls09:25:003
551799150,14cyclictest6012-21wc08:15:223
551799150,0cyclictest0-21swapper/310:41:093
551599159,5cyclictest31068-21grep11:25:142
551599156,8cyclictest0-21swapper/208:46:202
551599152,12cyclictest30732-21diskmemload09:17:162
5515991514,1cyclictest0-21swapper/210:47:132
5511991511,3cyclictest2904-21sh12:05:271
550899158,6cyclictest25929-21perf09:00:010
550899153,4cyclictest4576-21ssh10:50:140
550899153,4cyclictest4576-21ssh10:50:140
5517991410,3cyclictest7117-21tr11:30:143
5517991410,3cyclictest5182-21ssh09:13:513
5515991411,2cyclictest14462-21wc07:25:262
5515991411,2cyclictest14462-21wc07:25:262
551599140,3cyclictest0-21swapper/209:33:592
551599140,1cyclictest0-21swapper/211:50:552
551599140,13cyclictest0-21swapper/212:09:192
551599140,12cyclictest7082-21/usr/sbin/munin11:10:212
551199149,2cyclictest21515-21cut08:50:141
551199148,2cyclictest0-21swapper/111:15:141
551199145,7cyclictest30732-21diskmemload10:40:121
551199142,12cyclictest0-21swapper/111:23:361
551199141,4cyclictest0-21swapper/107:15:161
5511991413,1cyclictest0-21swapper/111:48:581
5511991410,2cyclictest10484-21grep09:35:261
551199140,1cyclictest0-21swapper/111:28:391
551199140,14cyclictest0-21swapper/109:27:281
551199140,12cyclictest0-21swapper/110:38:261
5508991413,1cyclictest13-21ksoftirqd/009:24:130
5508991413,1cyclictest13-21ksoftirqd/009:24:130
258312140,0chrt0-21swapper/210:24:062
551799139,3cyclictest7408-21sed07:10:273
551799139,3cyclictest24731-21grep09:25:163
551799139,3cyclictest12087-21tr10:35:133
551799139,2cyclictest779-21ssh09:50:013
551799134,5cyclictest225-21systemd-journal07:40:003
551799133,6cyclictest1142-21runrttasks12:14:163
5517991312,1cyclictest0-21swapper/312:25:303
5517991310,2cyclictest5466-21sed10:30:273
5517991310,2cyclictest4073-21grep08:10:273
5517991310,2cyclictest3106-21grep09:50:273
5517991310,2cyclictest23626-21cat08:55:003
5517991310,2cyclictest21865-21sshd07:40:273
5517991310,2cyclictest20334-21grep12:35:203
551599139,3cyclictest26044-21sh12:00:212
551599139,3cyclictest18839-21grep09:40:282
551599139,3cyclictest16066-21grep11:15:272
551599139,3cyclictest11602-21sed12:30:172
551599139,3cyclictest11602-21sed12:30:172
5515991312,1cyclictest0-21swapper/210:28:522
5515991310,2cyclictest6200-21wc08:15:262
5515991310,2cyclictest26893-21grep09:00:162
5515991310,2cyclictest21405-21head10:40:192
5515991310,2cyclictest1514-21cat11:45:272
5515991310,2cyclictest12935-21ssh10:55:132
5515991310,2cyclictest12138-21sed07:20:282
551599130,2cyclictest0-21swapper/210:50:142
551599130,2cyclictest0-21swapper/210:50:142
551599130,12cyclictest25048-21wc08:55:242
551199139,3cyclictest30072-21tr11:05:131
551199139,3cyclictest18296-21sed09:40:231
551199139,3cyclictest14007-21ls08:35:001
551199139,2cyclictest24727-21cron07:50:001
5511991312,1cyclictest0-21swapper/112:26:531
5511991310,2cyclictest6522-21grep11:10:171
5511991310,2cyclictest6378-21grep08:15:281
5511991310,2cyclictest3715-21grep10:10:201
5511991310,2cyclictest30732-21diskmemload10:05:151
5511991310,2cyclictest19803-21grep10:00:261
5511991310,2cyclictest18115-21grep12:15:141
5511991310,2cyclictest14048-21ls10:55:201
551199130,12cyclictest0-21swapper/109:12:471
550899134,8cyclictest0-21swapper/010:05:000
550899132,9cyclictest0-21swapper/012:39:530
5508991310,2cyclictest6936-21date09:15:000
5508991310,2cyclictest31260-21grep08:00:220
5508991310,2cyclictest12492-21cut10:35:160
550899130,12cyclictest17503-21cat10:00:010
551799129,2cyclictest6072-21grep10:50:233
551799129,2cyclictest6072-21grep10:50:233
551799129,2cyclictest29474-21cron08:00:003
551799129,2cyclictest25729-21taskset07:50:183
551799129,2cyclictest2173-21wc08:10:003
551799129,2cyclictest16357-21grep07:30:213
551799129,2cyclictest13617-21head08:30:273
551799129,2cyclictest12697-21grep12:30:273
551799129,2cyclictest12697-21grep12:30:263
551799129,2cyclictest12626-21ssh10:15:253
551799128,3cyclictest26805-21sed12:00:283
551799121,8cyclictest30046-21apt-get08:00:153
5517991210,1cyclictest208382sleep312:16:113
551799120,12cyclictest0-21swapper/312:06:343
551799120,11cyclictest23264-21grep07:45:163
551599129,2cyclictest8881-21sed07:15:212
551599129,2cyclictest8688-21sh11:30:252
551599129,2cyclictest4992-21wc10:30:232
551599129,2cyclictest4140-21wc12:25:272
551599129,2cyclictest31756-21grep08:00:272
551599129,2cyclictest31251-21wc11:05:212
551599129,2cyclictest29103-21grep09:05:152
551599129,2cyclictest28428-21seq11:42:342
551599129,2cyclictest28289-21grep12:20:282
551599129,2cyclictest27411-21grep07:55:162
551599129,2cyclictest22189-21sed11:00:172
551599129,2cyclictest20875-21cut07:40:172
551599129,2cyclictest17291-21sort08:40:172
551599128,3cyclictest0-21swapper/212:16:432
551599127,3cyclictest26623-21tr09:45:212
551599123,8cyclictest21341-21kworker/u8:010:05:162
551599120,11cyclictest0-21swapper/207:10:122
551199129,2cyclictest29319-21sort11:05:001
551199129,2cyclictest27103-21sh07:55:001
551199129,2cyclictest26459-21tr09:00:141
551199128,3cyclictest19322-21tr08:45:141
551199128,3cyclictest11268-21taskset08:25:281
551199128,2cyclictest29400-21grep10:25:261
551199127,4cyclictest8665-21awk09:15:211
551199127,4cyclictest7687-21ssh09:34:541
551199127,4cyclictest26527-21cat09:45:201
551199126,5cyclictest29489-21munin-run08:00:001
551199126,5cyclictest19975-21munin-run07:40:011
551199120,11cyclictest0-21swapper/108:20:131
550899129,2cyclictest8927-21tail11:50:170
550899129,2cyclictest7846-21sh07:15:000
550899129,2cyclictest6940-21tr11:30:130
550899129,2cyclictest26418-21wc07:50:280
550899129,2cyclictest20039-21chrt09:41:200
550899129,2cyclictest18886-21cat07:35:220
550899129,2cyclictest15772-21sh11:35:150
550899129,2cyclictest15142-21ssh12:32:190
550899129,2cyclictest15142-21ssh12:32:190
550899129,2cyclictest12630-21dump-pmu-power07:25:000
550899128,3cyclictest13-21ksoftirqd/010:40:140
5508991211,1cyclictest0-21swapper/009:00:010
551799119,1cyclictest8414-21tr07:15:173
551799119,1cyclictest194342sleep311:56:123
551799118,2cyclictest952-21cat11:45:203
551799118,2cyclictest7462-21ssh09:15:143
551799118,2cyclictest6988-21ntpq11:10:203
551799118,2cyclictest4894-21rm10:10:453
551799118,2cyclictest3742-21ssh11:08:533
551799118,2cyclictest3609-21seq11:27:523
551799118,2cyclictest28845-21tr09:05:153
551799118,2cyclictest27156-21grep12:20:183
551799118,2cyclictest26456-21ssh11:02:523
551799118,2cyclictest25451-21awk08:55:283
551799118,2cyclictest23401-21sh11:40:013
551799118,2cyclictest19463-21expr08:45:153
551799118,2cyclictest15083-21ssh11:35:063
551799118,2cyclictest12701-21ssh09:56:223
551799118,2cyclictest11814-21ssh11:51:483
551799117,3cyclictest7392-21tr08:20:143
551799110,10cyclictest27732-21ssh10:05:203
551599118,2cyclictest9772-21grep09:55:182
551599118,2cyclictest7393-21tr08:20:142
551599118,2cyclictest24302-21tail07:45:282
551599118,2cyclictest22596-21ssh11:20:152
551599118,2cyclictest21119-21sh11:57:312
551599118,2cyclictest17214-21chrt07:33:042
551599118,2cyclictest14244-21chrt10:16:332
551599118,2cyclictest13503-21sed08:30:272
5515991110,1cyclictest0-21swapper/208:50:022
551599110,10cyclictest26481-21rm12:39:382
551599110,10cyclictest0-21swapper/207:50:142
551199118,2cyclictest6939-21ssh10:31:351
551199118,2cyclictest6482-21head07:10:171
551199118,2cyclictest631-21cut08:05:161
551199118,2cyclictest30792-21head08:00:181
551199118,2cyclictest28785-21expr10:45:141
551199118,2cyclictest25702-21cut12:00:181
551199118,2cyclictest21080-21unixbench_multi10:20:271
551199118,2cyclictest16400-21awk08:40:001
551199118,2cyclictest14318-21ssh11:34:411
551199118,2cyclictest13629-21ssh11:53:131
551199117,2cyclictest12962-21grep07:25:151
551199117,2cyclictest12962-21grep07:25:151
551199116,4cyclictest14154-21sh12:12:311
551199111,1cyclictest22869-21rm11:39:481
550899119,2cyclictest648-21tune2fs08:05:160
550899119,1cyclictest5972-21sed08:15:220
550899119,1cyclictest341-21ls12:05:010
550899118,2cyclictest9686-21ssh12:29:360
550899118,2cyclictest8937-21wc07:15:200
550899118,2cyclictest3272-21ssh10:30:030
550899118,2cyclictest2998-21cut08:10:180
550899118,2cyclictest2989-21basename10:30:000
550899118,2cyclictest24482-21ssh10:23:020
550899118,2cyclictest24332-21basename12:00:000
550899118,2cyclictest21587-21grep11:00:140
550899118,2cyclictest18599-21grep12:15:160
550899118,2cyclictest16371-21ssh12:14:150
550899117,3cyclictest3550-21sh11:08:430
550899117,3cyclictest21514-21grep08:50:140
550899113,1cyclictest8778-21dpkg12:10:000
550899112,7cyclictest6750-21ssh11:10:180
550899111,8cyclictest32343-21/usr/sbin/munin09:30:230
550899110,0cyclictest0-21swapper/011:18:330
551799108,1cyclictest26394-21ls09:00:133
551799108,1cyclictest142712sleep308:35:123
551799108,1cyclictest124482sleep307:22:513
551799107,2cyclictest10966-21cat08:25:263
551799100,9cyclictest16995-21cat08:40:153
551599108,2cyclictest2715-21ssh10:10:142
551599108,1cyclictest9489-21chrt08:25:092
551599107,2cyclictest722-21tail08:05:162
551199108,1cyclictest24073-21cut08:55:161
551199108,1cyclictest11933-21switchtime07:20:271
551199107,2cyclictest22351-21cron07:45:001
551199107,2cyclictest17652-21cat08:40:191
551199107,2cyclictest16775-21timerandwakeup07:30:271
550899108,1cyclictest9711-21egrep08:25:150
550899108,1cyclictest7286-21wget08:20:140
550899108,1cyclictest28677-21ps07:55:250
550899108,1cyclictest26369-21ssh09:45:190
550899107,2cyclictest22306-21ssh11:20:060
550899107,2cyclictest18330-21sed08:40:260
550899107,2cyclictest12521-21grep08:30:170
550899103,4cyclictest24006-21diskmemload09:25:110
550899102,4cyclictest225-21systemd-journal07:45:010
550899100,9cyclictest10760-21ssh10:15:100
550899100,9cyclictest1034-21ssh09:50:060
55159997,1cyclictest15760-21sed08:35:262
55119997,1cyclictest3946-21ls08:10:271
55119997,1cyclictest304932sleep109:07:111
55089998,1cyclictest29876-21processes09:05:210
55089998,1cyclictest14273-21apt08:35:120
55089998,1cyclictest13470-21diskstats07:25:180
55089998,1cyclictest13470-21diskstats07:25:170
55089997,1cyclictest21013-21sed07:40:170
55089997,1cyclictest19363-21expr08:45:150
55089997,1cyclictest15233-21chrt07:30:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional