You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 22:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Tue Aug 11, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2183129581,9sleep00-21swapper/007:09:100
2188629364,10sleep30-21swapper/307:09:553
2169229267,9sleep20-21swapper/207:07:242
2181228265,13sleep10-21swapper/107:08:561
272482500,0sleep00-21swapper/011:32:370
171992310,0sleep00-21swapper/009:30:190
2198899290,1cyclictest0-21swapper/309:12:383
2198499280,24cyclictest0-21swapper/209:44:062
2198899250,2cyclictest0-21swapper/311:49:423
2198899240,0cyclictest0-21swapper/309:42:313
2198499240,23cyclictest0-21swapper/212:10:292
2198899231,0cyclictest0-21swapper/310:43:443
2198899230,4cyclictest0-21swapper/311:44:593
2198899230,1cyclictest0-21swapper/309:28:293
2198299230,3cyclictest0-21swapper/110:08:281
2198899220,1cyclictest0-21swapper/311:32:583
21982992220,1cyclictest103522chrt12:00:541
21978992218,3cyclictest13-21ksoftirqd/011:54:590
21978992217,2cyclictest141rcu_preempt12:00:290
21978992216,2cyclictest13-21ksoftirqd/010:33:120
21988992119,1cyclictest0-21swapper/311:10:143
2198899211,4cyclictest0-21swapper/311:28:103
2198899210,3cyclictest0-21swapper/309:32:423
2198899210,1cyclictest0-21swapper/309:16:353
2198499210,21cyclictest0-21swapper/209:33:222
21978992117,3cyclictest13-21ksoftirqd/010:40:140
21978992116,3cyclictest13-21ksoftirqd/010:51:080
2198899200,18cyclictest0-21swapper/312:33:333
2198499200,19cyclictest0-21swapper/209:13:302
2198299200,1cyclictest0-21swapper/109:37:241
2198299200,19cyclictest0-21swapper/112:05:171
2197899202,1cyclictest141rcu_preempt11:15:160
21988991917,1cyclictest2065-21taskset11:17:333
21982991918,1cyclictest0-21swapper/112:24:161
21982991917,1cyclictest23169-21taskset10:31:471
21982991915,4cyclictest0-21swapper/111:29:471
21978991912,2cyclictest13-21ksoftirqd/012:29:510
2197899190,1cyclictest141rcu_preempt10:22:250
2198899181,13cyclictest0-21swapper/310:00:153
2198499186,1cyclictest0-21swapper/212:15:312
2198499180,0cyclictest0-21swapper/210:42:102
21982991816,1cyclictest138422chrt09:47:511
21988991715,1cyclictest8593-21apt-get10:23:473
2198899170,1cyclictest0-21swapper/309:54:363
21984991717,0cyclictest0-21swapper/210:49:372
2198499170,14cyclictest0-21swapper/211:09:172
21982991715,1cyclictest50132chrt11:39:031
2198299170,16cyclictest0-21swapper/108:50:121
21978991714,2cyclictest13-21ksoftirqd/011:25:190
21982991615,1cyclictest0-21swapper/111:10:161
2198299161,1cyclictest14567-21diskmemload11:30:121
2198299160,16cyclictest0-21swapper/109:26:531
2197899169,2cyclictest13-21ksoftirqd/008:25:140
2197899163,4cyclictest9182-21awk09:25:200
21978991614,1cyclictest14438-21taskset10:45:440
21978991614,1cyclictest13579-21taskset10:25:570
21978991614,1cyclictest13579-21taskset10:25:560
21978991613,2cyclictest13-21ksoftirqd/012:35:460
21988991511,3cyclictest13263-21wc10:45:203
2198899150,0cyclictest0-21swapper/310:58:563
2198499150,0cyclictest0-21swapper/210:05:222
21982991514,1cyclictest8742-21awk11:40:271
21982991514,1cyclictest0-21swapper/112:25:131
2198299150,3cyclictest0-21swapper/111:09:171
2198299150,13cyclictest0-21swapper/111:47:571
2197899159,2cyclictest141rcu_preempt08:10:160
2197899156,8cyclictest161rcuc/010:55:570
21978991513,2cyclictest13-21ksoftirqd/009:57:120
21978991513,1cyclictest3872-21taskset09:22:130
21978991510,1cyclictest13-21ksoftirqd/011:41:540
2197899150,0cyclictest141rcu_preempt10:14:550
21988991410,3cyclictest7632-21grep11:20:263
21988991410,3cyclictest31082-21tr11:35:153
21988991410,3cyclictest2018-21wc10:20:003
2198899140,1cyclictest0-21swapper/312:26:313
2198899140,1cyclictest0-21swapper/312:13:383
2198899140,13cyclictest7527-21wc08:55:003
2198899140,12cyclictest5859-21grep07:40:243
2198499145,8cyclictest0-21swapper/210:04:062
2198499143,6cyclictest32474-21grep07:30:172
2198499141,9cyclictest0-21swapper/211:55:042
21984991412,1cyclictest0-21swapper/211:35:152
21984991410,3cyclictest7860-21cat11:40:192
2198499140,1cyclictest0-21swapper/211:12:262
21982991414,0cyclictest0-21swapper/111:04:291
21982991413,1cyclictest0-21swapper/107:40:121
21982991412,1cyclictest4581-21taskset10:01:211
21982991411,2cyclictest0-21swapper/110:48:501
21982991410,3cyclictest23245-21cat08:20:001
2198299140,13cyclictest0-21swapper/109:59:121
2197899146,5cyclictest13-21ksoftirqd/012:33:330
2197899145,1cyclictest141rcu_preempt09:19:370
21978991412,2cyclictest13-21ksoftirqd/009:13:140
21978991412,1cyclictest8802chrt10:38:110
21978991411,2cyclictest11226-21dump-pmu-power07:55:000
21978991410,3cyclictest9645-21sort12:20:160
2198899138,4cyclictest11038-21wc09:00:203
2198899131,0cyclictest0-21swapper/312:02:393
21988991310,2cyclictest8449-21wc12:20:013
21988991310,2cyclictest361irq_work/309:55:223
21988991310,2cyclictest13148-21cut09:05:163
21988991310,2cyclictest12328-21sed10:25:233
21988991310,2cyclictest12328-21sed10:25:233
2198499138,4cyclictest26445-21seq09:16:122
21984991310,2cyclictest9732-21grep12:00:282
21984991310,2cyclictest8187-21grep09:25:142
21984991310,2cyclictest23044-21cat11:50:002
2198299139,3cyclictest23697-21tr11:50:131
2198299132,8cyclictest0-21swapper/108:25:161
21982991312,1cyclictest0-21swapper/110:59:361
21982991310,2cyclictest16216-21cat09:10:201
21982991310,2cyclictest0-21swapper/109:15:051
2197899139,4cyclictest13-21ksoftirqd/007:15:150
2197899139,3cyclictest8169-21grep07:45:280
2197899139,3cyclictest3336-21grep10:00:280
2197899139,3cyclictest3263-21wc07:35:260
2197899139,3cyclictest13-21ksoftirqd/012:00:000
2197899138,4cyclictest13-21ksoftirqd/011:09:350
2197899131,1cyclictest141rcu_preempt10:09:170
21978991311,1cyclictest161rcuc/012:15:160
21978991311,1cyclictest13-21ksoftirqd/008:35:000
21978991310,2cyclictest18359-21sh08:10:000
2197899130,2cyclictest0-21swapper/009:41:030
2197899130,12cyclictest13147-21grep07:55:280
2198899129,2cyclictest5045-21grep11:00:163
2198899129,2cyclictest3246-21awk10:40:003
2198899129,2cyclictest32449-21sort07:30:173
2198899129,2cyclictest29669-21grep07:25:153
2198899129,2cyclictest29669-21grep07:25:153
2198899129,2cyclictest27468-21grep08:25:303
2198899129,2cyclictest24165-21sed08:20:173
2198899129,2cyclictest16489-21tr08:05:153
2198899129,2cyclictest15980-21cron08:05:003
2198899129,2cyclictest10824-21tail12:20:263
2198899129,2cyclictest10773-21tail07:50:273
2198899128,3cyclictest32269-21grep08:35:263
2198899128,3cyclictest1975-21cat08:40:253
21988991210,1cyclictest19405-21ls10:10:213
2198899120,1cyclictest0-21swapper/311:53:033
2198899120,0cyclictest0-21swapper/311:08:043
2198499129,2cyclictest822-21tr08:40:142
2198499129,2cyclictest7423-21basename09:25:002
2198499129,2cyclictest6913-21grep08:50:262
2198499129,2cyclictest31265-21sh11:55:002
2198499129,2cyclictest27214-21tr10:15:172
2198499129,2cyclictest26442-21grep08:25:202
2198499129,2cyclictest21255-21tail10:50:182
2198499129,2cyclictest20856-21ls10:30:222
2198499129,2cyclictest18262-21grep10:10:142
2198499129,2cyclictest17905-21sed08:05:262
2198499129,2cyclictest15521-21sed11:25:242
2198499129,2cyclictest13780-21cat11:25:002
2198499128,3cyclictest889-21idleruntime-cro10:00:002
2198499122,2cyclictest225-21systemd-journal12:24:552
21984991210,1cyclictest4872-21head07:40:142
2198499120,3cyclictest0-21swapper/212:37:132
2198499120,11cyclictest0-21swapper/208:15:152
2198299129,2cyclictest2931-21taskset11:18:141
2198299129,2cyclictest2503-21head07:35:171
2198299129,2cyclictest2336-21grep09:40:241
2198299129,2cyclictest22906-21cut07:10:161
2198299129,2cyclictest1883-21ntpq12:15:211
2198299128,4cyclictest25888-21apache210:25:591
2198299128,4cyclictest25888-21apache210:25:591
2198299128,3cyclictest32689-21cron08:39:591
2198299128,3cyclictest13583-21cron08:00:011
2198299127,4cyclictest5558-21cat10:40:261
2198299121,1cyclictest14567-21diskmemload09:54:141
21982991211,1cyclictest0-21swapper/110:23:151
2198299120,12cyclictest0-21swapper/110:35:181
2198299120,10cyclictest0-21swapper/111:55:161
2197899129,2cyclictest6245-21grep11:20:150
2197899129,2cyclictest27230-21grep10:15:180
2197899128,3cyclictest17062-21head11:45:280
2197899128,2cyclictest31618-21ls08:35:200
2197899128,2cyclictest14754-21cat09:10:000
2197899124,4cyclictest3419-21munin-node07:20:130
2197899123,1cyclictest141rcu_preempt12:10:130
2197899121,0cyclictest141rcu_preempt11:04:240
2198899118,2cyclictest8824-21ssh09:45:143
2198899118,2cyclictest7233-21tail07:45:183
2198899118,2cyclictest4047-21sh12:35:513
2198899118,2cyclictest28767-21sed07:20:263
2198899118,2cyclictest25813-21ssh10:33:533
2198899118,2cyclictest2303-21ssh11:56:113
2198899118,2cyclictest21813-21head10:50:233
2198899118,2cyclictest15958-21chrt12:05:103
2198899118,2cyclictest10822-21grep10:05:173
2198899115,2cyclictest225-21systemd-journal08:19:553
2198899111,7cyclictest39-21ksoftirqd/309:20:163
2198899110,10cyclictest25970-21grep07:15:223
2198499119,1cyclictest8950-21uname08:55:232
2198499119,1cyclictest76872sleep207:45:232
2198499118,2cyclictest9154-21rm09:45:162
2198499118,2cyclictest4545-21ssh11:00:072
2198499118,2cyclictest32488-21ssh10:37:322
2198499118,2cyclictest29617-21taskset11:15:062
2198499118,2cyclictest29599-21ssh09:37:512
2198499118,2cyclictest28532-21ssh10:55:022
2198499118,2cyclictest2624-21if_enp2s007:35:172
2198499118,2cyclictest25609-21aten2_r7power_c12:30:152
2198499118,2cyclictest24341-21awk07:15:002
2198499118,2cyclictest24068-21tune2fs08:20:162
2198499118,2cyclictest2252-21ssh10:20:102
2198499118,2cyclictest18289-21cat09:50:212
2198499118,2cyclictest17497-21sh12:25:142
2198499118,2cyclictest16407-21cut12:05:192
2198499118,2cyclictest15323-21sh10:27:202
2198499118,2cyclictest15323-21sh10:27:202
2198499118,2cyclictest14204-21expr08:00:152
2198499118,2cyclictest12319-21cat07:55:202
2198499117,3cyclictest30136-21grep07:25:172
2198499117,3cyclictest30136-21grep07:25:172
2198499110,10cyclictest26565-21taskset07:17:362
2198499110,10cyclictest18596-21acpi08:10:112
2198299119,1cyclictest26458-21head12:30:191
2198299119,1cyclictest1113-21grep09:20:231
2198299118,2cyclictest9375-21tr07:50:151
2198299118,2cyclictest3164-21tr08:45:141
2198299118,2cyclictest30182-21head07:25:181
2198299118,2cyclictest30182-21head07:25:181
2198299118,2cyclictest23883-21ssh09:34:561
2198299118,2cyclictest18852-21head10:10:171
2198299118,2cyclictest18613-21basename08:10:121
2198299113,4cyclictest0-21swapper/108:55:141
2197899118,2cyclictest5852-21grep07:40:240
2197899118,2cyclictest32503-21grep07:30:180
2197899118,2cyclictest3188-21tr08:45:140
2197899118,2cyclictest28645-21ssh09:37:060
2197899118,2cyclictest24295-21ssh11:11:080
2197899118,2cyclictest16208-21wget12:05:180
2197899118,2cyclictest1521-21wc08:40:190
2197899118,2cyclictest14581-21grep08:00:170
2197899118,2cyclictest12413-21ssh09:46:430
2197899118,2cyclictest11281-21cut09:00:220
2197899117,3cyclictest30115-21chrt07:25:170
2197899117,3cyclictest30115-21chrt07:25:160
2197899117,3cyclictest25097-21grep08:20:270
2197899116,1cyclictest13-21ksoftirqd/009:53:350
2197899115,3cyclictest13-21ksoftirqd/011:35:030
2198899108,1cyclictest78982sleep308:55:163
2198899108,1cyclictest3927-21sort08:45:193
2198899108,1cyclictest3054-21cut07:35:223
2198899108,1cyclictest30531-21date08:35:003
2198899108,1cyclictest22346-21cut07:10:143
2198899108,1cyclictest20368-21chrt08:10:283
2198899108,1cyclictest13423-21chrt07:58:023
2198899107,2cyclictest24319-21ssh09:35:063
2198499108,1cyclictest324372sleep208:36:512
2198499108,1cyclictest30021-21taskset08:31:192
2198499108,1cyclictest22435-21rm11:30:132
2198499108,1cyclictest10284-21grep09:00:142
2198499107,2cyclictest9374-21grep07:50:152
2198499107,2cyclictest27777-21head07:20:172
2198299108,2cyclictest9142-21ssh11:21:331
2198299108,2cyclictest2771-21ntp_kernel_err12:35:201
2198299108,1cyclictest28255-21aten2_r7power_e08:30:141
2198299108,1cyclictest16487-21grep08:05:151
2198299108,1cyclictest13204-21tail09:05:171
2198299108,1cyclictest10218-21tr09:00:151
2198299107,2cyclictest27528-21ssh12:11:151
2198299107,2cyclictest24101-21fw_packets08:20:171
2198299107,2cyclictest23489-21ssh10:51:191
2198299107,2cyclictest1700-21ntpq08:40:201
2198299102,7cyclictest8678-21latency_hist07:50:011
2198299100,1cyclictest0-21swapper/110:15:121
2197899108,1cyclictest7692-21apt-get08:55:140
2197899108,1cyclictest5338-21hostname08:50:130
2197899108,1cyclictest242302sleep007:13:590
2197899107,2cyclictest21042-21taskset08:15:140
219849998,1cyclictest124562sleep209:05:122
219849996,2cyclictest4093-21ntpq08:45:212
219829997,1cyclictest13836-21chrt08:00:031
219829996,2cyclictest27297-21tr07:20:141
219829987,1cyclictest32275-21expr07:30:161
219829986,1cyclictest26394-21seq07:15:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional