You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-01 - 16:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Aug 01, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1428529684,8sleep00-21swapper/007:09:210
1219829266,9sleep30-21swapper/307:05:023
1416329167,8sleep20-21swapper/207:07:462
1424428365,14sleep10-21swapper/107:08:501
326772650,0sleep10-21swapper/112:36:321
14428993130,1cyclictest0-21swapper/208:54:292
1442899300,29cyclictest0-21swapper/210:20:202
1442899270,27cyclictest0-21swapper/211:21:162
1443099251,14cyclictest0-21swapper/312:15:543
1442899254,5cyclictest141rcu_preempt11:49:442
1442899240,23cyclictest0-21swapper/212:09:342
1442899230,23cyclictest0-21swapper/211:40:592
1442899230,23cyclictest0-21swapper/211:15:372
1442499231,19cyclictest0-21swapper/109:00:141
1442199237,8cyclictest5266-21munin-run10:45:000
1443099222,8cyclictest16913-21/usr/sbin/munin11:10:223
14424992220,1cyclictest0-21swapper/112:30:151
1442199220,1cyclictest0-21swapper/011:07:430
1442899210,21cyclictest0-21swapper/211:13:102
1442899210,20cyclictest0-21swapper/211:25:472
1442499210,2cyclictest0-21swapper/111:58:051
141132210,0sleep30-21swapper/310:50:023
14428992011,6cyclictest0-21swapper/208:00:002
1442899200,20cyclictest0-21swapper/210:31:462
1442499200,19cyclictest0-21swapper/111:44:361
1443099190,15cyclictest24398-21sensors10:55:273
1442899199,9cyclictest26957-21apache209:39:052
1442899190,19cyclictest0-21swapper/211:05:052
1442899190,19cyclictest0-21swapper/209:16:352
1442899190,18cyclictest0-21swapper/212:34:072
1442199195,7cyclictest25119-21awk09:40:000
1442199190,14cyclictest0-21swapper/009:27:220
14430991814,4cyclictest0-21swapper/312:38:473
1442899181,2cyclictest0-21swapper/207:40:172
1442899180,18cyclictest0-21swapper/209:42:552
1442899180,16cyclictest0-21swapper/212:03:172
14424991818,0cyclictest0-21swapper/112:01:171
1442499181,1cyclictest31503-21nfs4_client08:50:201
1442499180,18cyclictest0-21swapper/109:18:381
1442499180,17cyclictest0-21swapper/110:31:491
1442199183,7cyclictest4855-21munin-run09:05:000
1442199181,1cyclictest14414-21cyclictest11:55:570
1443099172,3cyclictest0-21swapper/311:25:133
1443099170,14cyclictest0-21swapper/311:49:383
1442899170,17cyclictest0-21swapper/210:16:332
1442899170,17cyclictest0-21swapper/210:09:482
1442899170,17cyclictest0-21swapper/209:46:002
1442499170,14cyclictest0-21swapper/109:38:541
1442199175,10cyclictest0-21swapper/011:20:130
14430991615,1cyclictest0-21swapper/312:03:433
1442899167,8cyclictest32371-21idleruntime-cro09:24:592
14428991613,3cyclictest26972-21apache211:32:562
14424991614,2cyclictest38150irq/131-snd_hda11:22:131
14424991614,1cyclictest38150irq/131-snd_hda10:06:481
1442499160,1cyclictest0-21swapper/108:15:131
1442499160,15cyclictest0-21swapper/110:46:041
1442199163,7cyclictest26299-21munin-run07:34:590
14430991513,1cyclictest0-21swapper/309:10:243
14430991512,3cyclictest39-21ksoftirqd/309:19:333
1443099150,2cyclictest0-21swapper/312:29:373
1443099150,13cyclictest0-21swapper/311:20:413
1442899152,13cyclictest17471-21sh09:54:342
14428991513,1cyclictest0-21swapper/208:55:152
14428991512,2cyclictest30118-21sh08:50:002
1442899150,14cyclictest0-21swapper/212:28:292
1442899150,13cyclictest28433-21ssh10:00:212
14424991514,1cyclictest0-21swapper/110:15:441
14424991512,2cyclictest0-21swapper/108:20:131
1442199153,6cyclictest1285-21perf11:40:000
14430991412,2cyclictest0-21swapper/311:01:093
1443099140,2cyclictest0-21swapper/309:50:373
1443099140,13cyclictest0-21swapper/311:08:573
1443099140,12cyclictest4949-21head12:20:173
1443099140,0cyclictest0-21swapper/309:30:233
14428991411,2cyclictest9153-21wc09:10:212
1442899140,3cyclictest0-21swapper/209:28:572
1442499142,1cyclictest27138-21ssh09:20:541
14424991410,2cyclictest21159-21head12:10:231
1442499140,1cyclictest0-21swapper/108:10:271
1442499140,13cyclictest31110-21cron07:45:011
1442499140,12cyclictest0-21swapper/111:01:001
14421991410,3cyclictest6586-21cron11:05:000
14421991410,3cyclictest25290-21wc08:40:000
1442199140,8cyclictest15635-21perf09:15:000
1442199140,1cyclictest0-21swapper/009:17:530
246392130,0chrt0-21swapper/011:53:380
208542130,2chrt20852-21turbostat.cron10:35:000
1443099139,3cyclictest18029-21wc08:25:013
1443099139,2cyclictest11110-21grep08:10:153
1443099136,4cyclictest473-21apt-get09:25:153
1443099133,4cyclictest15558-21cron08:20:003
1443099132,10cyclictest25078-21sh08:40:003
1443099131,9cyclictest1566-21apache_accesses09:45:113
14430991313,0cyclictest0-21swapper/312:08:443
14430991310,2cyclictest31846-21tr07:45:153
14430991310,2cyclictest26237-21wc11:15:253
14430991310,2cyclictest24057-21cat07:30:013
14430991310,2cyclictest20358-21grep12:10:163
14430991310,2cyclictest19263-21grep11:30:263
1443099130,12cyclictest28234-21tr08:45:153
1443099130,10cyclictest0-21swapper/310:02:553
1442899139,2cyclictest28127-21wc07:35:262
14428991310,2cyclictest9446-21sed08:05:182
14428991310,2cyclictest8948-21basename09:30:142
14428991310,2cyclictest7021-21ssh10:45:202
14428991310,2cyclictest30770-21cat11:00:002
14428991310,2cyclictest30123-21grep12:15:272
14428991310,2cyclictest29591-21tr12:35:152
14428991310,2cyclictest23124-21fgrep07:25:222
14428991310,2cyclictest19851-21sed08:25:272
14428991310,2cyclictest1246-21sh11:40:002
1442899130,2cyclictest0-21swapper/210:25:172
1442499139,3cyclictest27835-21sed07:35:221
1442499139,3cyclictest25901-21tr09:40:141
1442499139,3cyclictest23236-21grep08:35:151
1442499139,3cyclictest14510-21sed12:25:281
1442499139,2cyclictest22076-21grep07:25:141
14424991310,2cyclictest31829-21tr07:45:151
1442499130,13cyclictest0-21swapper/111:06:121
1442199139,2cyclictest10362-21ls11:25:210
1442199131,6cyclictest27927-21apt-get10:20:000
14421991313,0cyclictest0-21swapper/012:14:280
14421991310,2cyclictest7452-21cat10:45:260
14421991310,2cyclictest32071-21grep08:50:270
14421991310,2cyclictest29816-21ssh10:00:550
14421991310,2cyclictest29623-21grep08:45:250
14421991310,2cyclictest12173-21grep08:10:220
1442199130,1cyclictest0-21swapper/011:16:350
1442199130,12cyclictest1043-21grep08:55:180
1443099129,2cyclictest7678-21sed10:45:263
1443099129,2cyclictest6832-21head09:05:263
1443099129,2cyclictest31055-21ssh10:40:213
1443099129,2cyclictest30493-21grep08:50:143
1443099129,2cyclictest27611-21wc08:45:003
1443099129,2cyclictest26476-21grep11:35:163
1443099129,2cyclictest20101-21cut07:20:153
1443099129,2cyclictest19404-21tail11:50:173
1443099128,3cyclictest15323-21ssh10:30:423
1443099122,6cyclictest21488-21df_abs12:30:163
1442899129,2cyclictest8020-21grep08:00:282
1442899129,2cyclictest31755-21sed10:40:272
1442899129,2cyclictest19159-21sh11:50:152
1442899129,2cyclictest16272-21grep07:10:262
1442899129,2cyclictest15563-21apt08:20:002
1442899125,6cyclictest26972-21apache211:57:582
1442899123,2cyclictest19745-21apache_volume12:10:132
1442499129,3cyclictest55650irq/127-enp2s0-10:50:191
1442499129,2cyclictest9573-21grep09:30:171
1442499129,2cyclictest6218-21grep09:05:191
1442499129,2cyclictest575-21sh09:25:151
1442499129,2cyclictest30148-21wc10:20:241
1442499129,2cyclictest29711-21head08:45:251
1442499129,2cyclictest28230-21tr12:15:141
1442499129,2cyclictest2488-21sleep09:00:001
1442499129,2cyclictest22691-21tr10:35:251
1442499129,2cyclictest21142-21sed07:20:251
1442499129,2cyclictest19083-21grep09:55:181
1442499129,2cyclictest18635-21basename11:50:141
1442499126,4cyclictest26278-21cron07:34:591
14424991211,1cyclictest0-21swapper/109:54:301
1442499120,11cyclictest9412-21tail08:05:181
1442499120,11cyclictest16537-21grep11:10:201
1442199129,2cyclictest9460-21sed08:05:180
1442199129,2cyclictest9442-21df09:30:160
1442199129,2cyclictest5535-21grep12:20:220
1442199129,2cyclictest4946-21wc12:40:010
1442199129,2cyclictest4832-21grep12:00:260
1442199129,2cyclictest3525-21tail09:45:260
1442199129,2cyclictest31845-21grep07:45:150
1442199129,2cyclictest20723-21grep09:55:290
1442199129,2cyclictest19853-21grep08:25:270
1442199129,2cyclictest19362-21sed11:30:280
1442199129,2cyclictest15815-21grep10:50:240
1442199128,2cyclictest19139-21cron07:20:010
14421991210,1cyclictest15281-21cut07:10:170
1442199120,11cyclictest7217-21cron09:10:000
1442199120,11cyclictest25317-21grep09:20:160
1443099119,1cyclictest17552-21expr09:35:143
1443099118,2cyclictest7727-21ssh10:26:183
1443099118,2cyclictest5605-21ssh11:41:523
1443099118,2cyclictest4863-21ls09:05:013
1443099118,2cyclictest28874-21egrep10:20:163
1443099118,2cyclictest28415-21taskset07:36:113
1443099118,2cyclictest27851-21expr11:55:193
1443099118,2cyclictest27280-21ssh09:40:243
1443099118,2cyclictest26268-21taskset07:34:533
1443099118,2cyclictest26061-21ssh09:20:223
1443099118,2cyclictest25238-21ssh10:36:553
1443099118,2cyclictest22790-21sed08:35:003
1443099118,2cyclictest20551-21grep09:55:283
1443099118,2cyclictest19759-21chrt10:15:043
1443099118,2cyclictest15858-21cut07:10:223
1443099118,2cyclictest12568-21ssh10:10:203
1443099113,3cyclictest7226-21diskmemload10:08:083
1442899118,2cyclictest4255-21cut12:20:152
1442899118,2cyclictest3621-21taskset07:54:402
1442899118,2cyclictest31933-21sort11:00:172
1442899118,2cyclictest31463-21sed07:45:002
1442899118,2cyclictest22315-21ssh10:35:212
1442899118,2cyclictest21727-21sh10:54:382
1442899118,2cyclictest20264-21ps09:55:242
1442899118,2cyclictest16141-21cut08:20:142
1442899118,2cyclictest15020-21ssh10:11:332
1442899117,3cyclictest20420-21wc07:20:182
1442899117,2cyclictest17260-21tr07:15:132
1442499118,2cyclictest9238-21sort11:25:141
1442499118,2cyclictest6057-21basename08:00:011
1442499118,2cyclictest5263-21sh10:25:161
1442499118,2cyclictest31156-21ssh10:40:221
1442499118,2cyclictest28097-21sed10:00:191
1442499118,2cyclictest22256-21sed08:30:271
1442499118,2cyclictest21209-21sh11:31:511
1442499118,2cyclictest1606-21ssh09:45:131
1442499118,2cyclictest12381-21cut10:10:191
1442499118,2cyclictest10974-21ssh11:45:191
1442499118,2cyclictest10758-21ssh12:24:031
1442499118,2cyclictest10024-21ssh09:10:391
1442499117,3cyclictest26495-21sed11:15:271
1442499116,2cyclictest8424-21cron08:05:001
1442499113,7cyclictest16754-21cron07:15:001
1442499113,4cyclictest225-21systemd-journal07:52:431
1442199119,1cyclictest11357-21ssh12:05:130
1442199118,2cyclictest6557-21grep08:00:160
1442199118,2cyclictest5443-21head10:25:170
1442199118,2cyclictest5337-21rm11:41:380
1442199118,2cyclictest3935-21sh10:05:170
1442199118,2cyclictest30257-21chrt12:15:300
1442199118,2cyclictest29260-21cron10:40:000
1442199118,2cyclictest28319-21ssh10:20:010
1442199118,2cyclictest27521-21chrt08:44:570
1442199118,2cyclictest25413-21ssh09:40:050
1442199118,2cyclictest23775-21ssh11:14:580
1442199118,2cyclictest23180-21expr10:55:170
1442199118,2cyclictest20847-21cat07:20:240
1442199118,2cyclictest18486-21ssh10:14:160
1442199118,2cyclictest16666-21sort08:20:160
1442199118,2cyclictest1658-21cut07:50:150
1442199118,2cyclictest14331-21ssh09:52:060
1442199117,3cyclictest10873-21sort11:45:180
1442199114,6cyclictest22720-21munin-run08:35:000
1442199110,1cyclictest0-21swapper/012:30:140
1443099108,1cyclictest8936-21cut08:05:163
1443099108,1cyclictest6292-21taskset08:00:133
1443099108,1cyclictest1139-21hostname08:55:193
1443099107,2cyclictest5057-21tr07:55:193
1443099107,2cyclictest29843-21cat07:40:213
1443099107,2cyclictest18775-21grep08:25:163
1443099107,2cyclictest17000-21taskset07:15:073
1443099107,2cyclictest1556-21apache_volume07:50:143
1442899108,1cyclictest20601-21apache_volume08:30:132
1442899108,1cyclictest13363-21wc08:15:002
1442899107,2cyclictest5349-21grep09:05:142
1442899107,2cyclictest23404-21expr08:35:152
1442499108,1cyclictest28160-21sh11:35:421
1442499107,3cyclictest23402-21ssh10:55:181
1442499107,2cyclictest25324-21sshd08:40:021
1442499107,2cyclictest13558-21ssh12:05:361
1442499101,6cyclictest20340-21dump-pmu-power08:30:001
1442499100,9cyclictest18619-21cat07:15:251
1442199108,1cyclictest12335-21ssh12:25:060
1442199107,2cyclictest5516-21cat07:55:260
1442199107,2cyclictest30002-21cat07:40:220
1442199107,2cyclictest27333-21head07:35:170
1442199107,2cyclictest22792-21cat07:25:190
1442199107,2cyclictest13448-21apache_volume08:15:120
144289997,1cyclictest272682sleep208:41:422
144289997,1cyclictest241892sleep207:30:142
144289996,2cyclictest3505-21tail09:00:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional