You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-18 - 04:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Apr 18, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
490621220,1sleep2311rcuc/223:07:342
693229667,6sleep20-21swapper/219:06:402
712829565,11sleep30-21swapper/319:09:143
698529582,9sleep00-21swapper/019:07:220
716328365,14sleep10-21swapper/119:09:401
38212620,1sleep33820-21ssh21:30:213
726399240,23cyclictest0-21swapper/023:08:490
200172220,0sleep00-21swapper/022:38:070
727099210,19cyclictest0-21swapper/121:44:171
727099210,0cyclictest0-21swapper/123:40:101
7263992120,1cyclictest0-21swapper/000:30:070
7263992120,1cyclictest0-21swapper/000:30:060
7263992119,1cyclictest0-21swapper/022:48:060
726399210,19cyclictest0-21swapper/023:40:130
727899200,1cyclictest0-21swapper/320:00:133
727599200,17cyclictest0-21swapper/222:54:412
727899197,8cyclictest19293-21runrttasks23:15:433
726399190,19cyclictest0-21swapper/000:38:270
727899180,1cyclictest0-21swapper/319:30:133
7275991818,0cyclictest0-21swapper/222:07:252
7275991814,3cyclictest0-21swapper/222:16:112
727599180,3cyclictest0-21swapper/223:29:112
727099180,2cyclictest0-21swapper/122:00:541
726399182,2cyclictest141rcu_preempt21:30:150
726399180,6cyclictest0-21swapper/023:04:220
76522170,0sleep00-21swapper/021:14:030
7278991715,2cyclictest0-21swapper/321:05:123
7275991715,1cyclictest0-21swapper/222:37:522
727599170,1cyclictest0-21swapper/223:42:362
727099174,13cyclictest31695-21sh00:20:461
7263991714,2cyclictest13-21ksoftirqd/023:48:450
727599163,9cyclictest17071-21ssh21:19:442
727599160,15cyclictest0-21swapper/200:29:102
7263991611,3cyclictest7890-21ls22:30:240
726399160,0cyclictest0-21swapper/023:20:250
7278991514,1cyclictest0-21swapper/322:10:193
7278991512,2cyclictest39-21ksoftirqd/322:58:233
727899150,0cyclictest0-21swapper/321:47:543
727599153,9cyclictest9278-21cups-browsed00:19:592
727599151,1cyclictest0-21swapper/223:58:442
727599150,15cyclictest0-21swapper/222:25:242
727599150,13cyclictest0-21swapper/219:50:132
727099151,3cyclictest0-21swapper/100:00:141
7270991513,2cyclictest0-21swapper/123:51:161
727099151,2cyclictest10287-21rm21:54:071
727099151,13cyclictest11931-21ssh21:15:431
7270991510,5cyclictest38150irq/131-snd_hda23:19:051
727099150,2cyclictest0-21swapper/119:35:121
727099150,0cyclictest0-21swapper/123:00:541
726399154,5cyclictest14028-21hddtemp_smartct22:15:170
7263991514,1cyclictest0-21swapper/021:20:250
726399150,15cyclictest0-21swapper/023:34:080
727899147,3cyclictest13336-21cron00:30:003
7278991410,3cyclictest11467-21grep21:15:273
727899140,13cyclictest0-21swapper/300:01:053
727599149,4cyclictest1427-21chrt00:22:282
727599147,1cyclictest24774-21latency_hist23:20:002
7275991411,3cyclictest32-21ksoftirqd/223:48:112
7275991410,3cyclictest25554-21tr23:20:142
7275991410,3cyclictest16347-21sed22:55:162
727599140,1cyclictest0-21swapper/221:55:112
727599140,12cyclictest0-21swapper/221:24:172
727099149,5cyclictest24565-21ssh22:40:221
727099142,1cyclictest448-21diskmemload21:31:011
727099140,13cyclictest13808-21cron20:29:591
7263991413,1cyclictest0-21swapper/023:59:370
7263991410,3cyclictest28787-21ssh00:00:170
726399140,8cyclictest0-21swapper/021:55:050
726399140,2cyclictest0-21swapper/022:10:120
727899139,3cyclictest10203-21grep19:15:143
727899139,2cyclictest24214-21grep22:40:193
727899137,3cyclictest1997-21sed23:05:263
727899136,3cyclictest225-21systemd-journal00:06:263
727899132,9cyclictest448-21diskmemload00:15:163
7278991310,2cyclictest831-21sed22:45:253
7278991310,2cyclictest8156-21tr22:50:163
7278991310,2cyclictest30838-21sed22:25:173
7278991310,2cyclictest26197-21cut23:20:163
7278991310,2cyclictest24085-21grep20:50:133
7278991310,2cyclictest23699-21wc00:35:223
7278991310,2cyclictest14818-21grep20:30:173
7278991310,2cyclictest11512-21cut23:50:133
727899130,12cyclictest0-21swapper/321:50:143
727599139,3cyclictest24422-21tr23:00:142
727599138,4cyclictest15044-21ls20:30:192
727599137,5cyclictest8304-21runrttasks22:11:372
7275991312,1cyclictest767-21ssh21:29:122
7275991312,1cyclictest0-21swapper/221:32:212
7275991310,2cyclictest30187-21wc00:40:002
7275991310,2cyclictest19040-21chrt23:54:582
7275991310,2cyclictest18094-21grep20:35:252
727599130,13cyclictest0-21swapper/222:04:522
727599130,11cyclictest23213-21wc22:20:252
727099135,2cyclictest990-21ssh21:10:121
727099133,8cyclictest448-21diskmemload00:00:001
727099131,1cyclictest5552-21/usr/sbin/munin00:05:211
7270991311,1cyclictest9056-21rm23:48:441
7270991311,1cyclictest32681-21tr22:45:191
7270991310,2cyclictest28711-21grep21:00:141
7270991310,2cyclictest18737-21sh21:20:161
7270991310,2cyclictest0-21swapper/100:13:161
726399139,3cyclictest27729-21grep20:55:250
726399139,3cyclictest23039-21grep19:40:160
726399139,3cyclictest22880-21grep20:45:250
726399139,3cyclictest20554-21grep20:40:270
726399134,8cyclictest537-21snmpd21:18:230
726399131,6cyclictest1726-21perf20:04:590
726399131,0cyclictest141rcu_preempt00:14:540
7263991310,2cyclictest7627-21tr22:50:140
7263991310,2cyclictest29791-21sed00:20:150
7263991310,2cyclictest27251-21sed21:25:170
7263991310,2cyclictest17173-21grep22:55:220
726399130,1cyclictest0-21swapper/000:06:280
239262130,2sleep30-21swapper/322:20:443
727899129,2cyclictest9343-21chrt20:20:133
727899129,2cyclictest8494-21wc19:10:183
727899129,2cyclictest7182-21tr20:15:133
727899129,2cyclictest3225-21tail23:25:253
727899129,2cyclictest27506-21tr19:50:143
727899129,2cyclictest26094-21grep23:00:263
727899129,2cyclictest22443-21cron22:40:003
727899129,2cyclictest20329-21tr19:35:143
727899129,2cyclictest18778-21grep23:35:143
727899129,2cyclictest18761-21head21:20:163
727899129,2cyclictest160692sleep300:11:393
727899129,2cyclictest14183-21ssh22:15:173
727899129,2cyclictest12030-21cut21:55:123
727899129,2cyclictest11578-21wc23:30:263
727899129,2cyclictest11448-21grep21:35:173
727899128,3cyclictest27219-21tr23:40:153
727899128,3cyclictest21065-21grep21:40:243
727899128,3cyclictest15798-21grep19:25:173
727899128,2cyclictest28248-21sed21:25:263
727899124,4cyclictest225-21systemd-journal21:00:003
727899120,11cyclictest0-21swapper/300:30:033
727899120,11cyclictest0-21swapper/300:30:033
727599129,2cyclictest8988-21cat19:10:242
727599129,2cyclictest29428-21cat21:45:262
727599129,2cyclictest23064-21grep19:40:162
727599129,2cyclictest22766-21cat20:45:252
727599129,2cyclictest20362-21cat20:40:262
727599129,2cyclictest12770-21tr19:20:122
727599129,2cyclictest12484-21diskmemload00:10:112
727599129,2cyclictest11992-21cut20:25:132
727599129,2cyclictest11443-21idleruntime-cro20:25:002
727599128,3cyclictest21737-21grep19:35:252
727599123,7cyclictest0-21swapper/223:40:002
727599122,8cyclictest10745-21ssh21:35:142
727599121,4cyclictest0-21swapper/222:30:122
7275991210,1cyclictest448-21diskmemload23:12:492
727099129,2cyclictest8109-21grep22:30:251
727099129,2cyclictest7209-21tr20:15:131
727099129,2cyclictest6020-21grep22:10:201
727099129,2cyclictest29687-21grep22:05:171
727099129,2cyclictest23733-21grep00:35:221
727099129,2cyclictest22307-21grep00:15:221
727099129,2cyclictest21901-21sort20:45:161
727099129,2cyclictest14809-21head20:30:171
727099128,3cyclictest792-21cron23:25:001
727099127,2cyclictest12603-21ssh22:52:571
727099127,2cyclictest0-21swapper/123:37:441
727099126,2cyclictest225-21systemd-journal20:39:591
727099123,5cyclictest13172-21fschecks_time19:20:151
727099120,9cyclictest10615-21ssh00:27:531
727099120,11cyclictest15167-21seq21:56:221
726399129,2cyclictest5696-21cut00:25:160
726399129,2cyclictest28884-21cut22:05:140
726399129,2cyclictest24960-21head22:40:260
726399129,2cyclictest18296-21sort19:30:160
726399129,2cyclictest16692-21grep19:25:250
726399129,2cyclictest13167-21ssh23:12:270
726399129,2cyclictest11359-21wc19:15:220
726399129,2cyclictest10765-21ssh21:54:290
726399128,3cyclictest14828-21grep20:30:170
726399128,2cyclictest24305-21grep20:50:130
726399128,2cyclictest11955-21tr20:25:120
726399122,1cyclictest225-21systemd-journal23:40:000
7263991212,0cyclictest0-21swapper/022:24:110
726399120,1cyclictest0-21swapper/023:27:290
726399120,0cyclictest0-21swapper/021:49:580
726399120,0cyclictest0-21swapper/021:40:520
727899119,1cyclictest17081-21sort20:35:163
727899118,2cyclictest5249-21grep20:10:163
727899118,2cyclictest4521-21ssh21:11:373
727899118,2cyclictest3635-21head23:45:163
727899118,2cyclictest2404-21expr20:05:143
727899118,2cyclictest23839-21ssh22:01:353
727899118,2cyclictest22445-21apt-get19:40:133
727899118,2cyclictest12635-21grep19:20:123
727899118,2cyclictest11932-21cut20:25:133
727899118,2cyclictest10236-21ssh22:32:023
727899110,8cyclictest0-21swapper/323:10:023
727899110,7cyclictest29655-21cut19:55:143
727899110,10cyclictest0-21swapper/319:45:123
727599118,2cyclictest8620-21grep20:15:252
727599118,2cyclictest5443-21wc00:05:212
727599118,2cyclictest4344-21taskset21:50:172
727599118,2cyclictest4122-21ssh22:47:522
727599118,2cyclictest29966-21sed00:00:272
727599118,2cyclictest25261-21grep20:50:232
727599118,2cyclictest19375-21sed19:30:262
727599118,2cyclictest15473-21expr19:25:152
727599118,2cyclictest13407-21rm23:31:372
727599114,3cyclictest1290-21tr21:10:142
727599110,1cyclictest448-21diskmemload00:30:532
727599110,1cyclictest448-21diskmemload00:30:532
727099119,1cyclictest27168-21wc19:50:001
727099119,1cyclictest25309-21grep20:50:231
727099118,2cyclictest9406-21tr20:20:141
727099118,2cyclictest7658-21ssh23:09:421
727099118,2cyclictest5278-21head20:10:161
727099118,2cyclictest3631-21ssh23:25:391
727099118,2cyclictest32519-21sed21:05:261
727099118,2cyclictest30784-21grep22:25:171
727099118,2cyclictest27486-21ssh21:45:071
727099118,2cyclictest2663-21tune2fs20:05:161
727099118,2cyclictest26292-21tr20:55:141
727099118,2cyclictest19497-21ssh22:18:491
727099118,2cyclictest18271-21uniq19:30:161
727099118,2cyclictest16495-21tune2fs22:55:171
727099118,2cyclictest14842-21sh23:13:471
727099118,2cyclictest13774-21ssh00:30:101
727099118,2cyclictest13774-21ssh00:30:091
727099118,2cyclictest13037-21ssh21:35:411
727099118,2cyclictest11353-21wc23:30:221
727099117,3cyclictest21290-21rm22:20:001
727099117,3cyclictest16874-21ssh22:35:401
727099117,1cyclictest29750-21ssh21:26:301
727099115,1cyclictest225-21systemd-journal19:25:031
726399118,2cyclictest9935-21cut20:20:170
726399118,2cyclictest31974-21ssh22:25:260
726399118,2cyclictest31735-21turbostat.cron20:00:000
726399118,2cyclictest31513-21head21:05:160
726399118,2cyclictest20933-21rm00:15:140
726399118,2cyclictest17523-21rm23:53:480
726399113,3cyclictest17212-21tr23:15:130
726399110,10cyclictest10545-21taskset21:35:050
727899108,1cyclictest22186-21ls20:45:173
727899107,2cyclictest29507-21sort22:05:153
727899107,2cyclictest19494-21grep20:40:173
727899106,3cyclictest29677-21wc21:00:223
727899101,8cyclictest19625-21ssh23:55:133
727899100,9cyclictest29129-21ssh00:20:083
727599108,1cyclictest267872sleep219:47:382
727599108,1cyclictest2435-21cat20:05:142
727599108,1cyclictest21456-21ssh21:40:412
727599108,1cyclictest100662sleep219:15:092
727599107,2cyclictest4817-21cut20:10:142
727599107,2cyclictest32198-21chrt21:05:242
727599107,2cyclictest31314-21grep19:55:262
727599107,2cyclictest23236-21cut22:40:132
727599106,3cyclictest29193-21tail21:00:172
727599102,2cyclictest225-21systemd-journal20:01:102
727599100,9cyclictest26446-21expr20:55:142
727099108,1cyclictest29681-21cut19:55:141
727099108,1cyclictest19428-21df20:40:161
727099108,1cyclictest10374-21expr19:15:151
727099107,2cyclictest22630-21awk19:40:141
727099102,4cyclictest225-21systemd-journal19:54:281
726399108,1cyclictest94852sleep019:13:270
726399108,1cyclictest26946-21chrt19:49:410
726399108,1cyclictest20250-21cat19:35:130
726399108,1cyclictest17114-21grep20:35:160
726399107,2cyclictest4605-21cut20:10:140
726399107,2cyclictest20356-21grep22:00:110
726399102,7cyclictest14136-21unixbench_multi19:20:250
726399100,9cyclictest7363-21expr20:15:140
726399100,9cyclictest2771-21sed20:05:170
72709997,1cyclictest32278-21expr20:00:131
72709996,2cyclictest8737-21head19:10:201
72639997,1cyclictest29020-21grep21:00:150
72639995,3cyclictest27474-21grep19:50:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional