You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 08:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Apr 08, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13684210174,8sleep20-21swapper/219:09:522
1348928859,9sleep30-21swapper/319:07:213
1349128471,9sleep00-21swapper/019:07:210
1353427958,6sleep10-21swapper/119:07:541
281152620,0sleep10-21swapper/123:17:011
1378399271,1cyclictest25364-21sed19:30:271
1378399270,2cyclictest23270-21cron21:19:591
13788992625,1cyclictest0-21swapper/323:13:253
1378599260,26cyclictest0-21swapper/222:23:152
1378399241,1cyclictest22712-21ssh22:55:121
1378399231,17cyclictest0-21swapper/100:22:091
13788992222,0cyclictest0-21swapper/300:22:193
1378899220,21cyclictest0-21swapper/323:20:353
1378599229,2cyclictest141rcu_preempt23:53:492
13783992221,1cyclictest0-21swapper/122:34:431
13783992219,2cyclictest0-21swapper/122:06:071
1378599210,20cyclictest0-21swapper/221:15:172
1378399212,19cyclictest6337-21diskmemload23:45:491
1378399210,20cyclictest0-21swapper/121:30:141
1378899201,1cyclictest0-21swapper/322:58:583
13783992019,1cyclictest0-21swapper/120:22:021
1378399192,16cyclictest21304-21apt-get22:35:121
1378399190,18cyclictest0-21swapper/122:40:131
1377899190,19cyclictest0-21swapper/023:30:480
1378899181,8cyclictest6337-21diskmemload23:56:173
1378899180,17cyclictest0-21swapper/322:13:423
1378599180,13cyclictest0-21swapper/219:45:142
13783991816,1cyclictest0-21swapper/121:49:001
1378399180,17cyclictest0-21swapper/122:03:081
1378399180,17cyclictest0-21swapper/122:03:081
1377899189,2cyclictest13-21ksoftirqd/022:50:000
1378899172,2cyclictest225-21systemd-journal22:54:593
1378399175,11cyclictest1142-21runrttasks22:49:071
13783991716,1cyclictest25-21ksoftirqd/121:43:081
13783991713,2cyclictest0-21swapper/100:31:291
1378399170,17cyclictest0-21swapper/100:27:431
1377899172,11cyclictest16929-21/usr/sbin/munin21:35:110
13788991615,1cyclictest0-21swapper/322:05:133
13788991615,0cyclictest0-21swapper/300:39:003
13788991614,1cyclictest0-21swapper/322:49:443
1378899160,2cyclictest0-21swapper/323:00:133
1378899160,0cyclictest0-21swapper/300:32:183
1378599160,15cyclictest0-21swapper/222:29:582
1378599160,14cyclictest0-21swapper/200:20:282
1378399169,4cyclictest0-21swapper/121:38:211
1378399160,16cyclictest0-21swapper/121:58:001
1378399160,0cyclictest0-21swapper/123:57:431
1377899165,6cyclictest225-21systemd-journal00:15:010
1377899160,15cyclictest0-21swapper/000:02:460
13788991514,1cyclictest0-21swapper/323:19:533
1378899150,15cyclictest0-21swapper/321:57:203
1378899150,14cyclictest0-21swapper/323:26:363
13785991515,0cyclictest0-21swapper/223:22:312
1378599150,15cyclictest0-21swapper/221:35:592
1378399153,11cyclictest7835-21dump-pmu-power20:05:011
1378399153,11cyclictest7835-21dump-pmu-power20:05:001
13783991515,0cyclictest0-21swapper/122:29:131
13778991513,1cyclictest13-21ksoftirqd/022:11:340
13778991511,3cyclictest24140-21tr23:15:140
1377899150,1cyclictest0-21swapper/023:36:330
1378899143,10cyclictest11389-21ssh00:05:153
13788991414,0cyclictest0-21swapper/322:03:573
13788991414,0cyclictest0-21swapper/322:03:563
13788991410,3cyclictest20695-21head20:30:173
1378899140,0cyclictest0-21swapper/300:16:183
13785991410,3cyclictest27825-21sh00:14:552
1378399142,6cyclictest6337-21diskmemload23:01:341
13783991413,1cyclictest0-21swapper/122:50:321
13783991413,1cyclictest0-21swapper/100:17:571
13783991412,1cyclictest0-21swapper/100:02:321
1378399141,1cyclictest6337-21diskmemload21:11:201
1378399140,1cyclictest0-21swapper/121:23:481
1377899147,6cyclictest22191-21awk22:55:000
1377899142,7cyclictest6337-21diskmemload00:40:000
1377899142,4cyclictest0-21swapper/022:20:160
13778991410,3cyclictest16115-21cron19:15:000
1377899140,1cyclictest0-21swapper/022:07:050
1377899140,1cyclictest0-21swapper/000:17:450
1377899140,14cyclictest0-21swapper/023:24:130
1377899140,14cyclictest0-21swapper/021:20:590
1378899139,3cyclictest23077-21grep19:25:273
1378899135,6cyclictest18247-21ssh22:33:033
13788991311,1cyclictest0-21swapper/300:28:073
13788991310,2cyclictest8064-21grep23:05:173
13788991310,2cyclictest3996-21grep19:55:183
13788991310,2cyclictest25790-21grep23:35:153
13788991310,2cyclictest23441-21cat21:20:013
13788991310,2cyclictest21773-21wc00:10:243
13788991310,2cyclictest16617-21cut19:15:143
13788991310,2cyclictest1161-21grep19:50:133
13788991310,2cyclictest10651-21grep23:45:163
1378899130,9cyclictest17894-21smartctl23:30:153
1378599139,3cyclictest22857-21cat19:25:262
1378599139,3cyclictest18848-21ssh00:09:352
13785991310,2cyclictest28760-21grep19:40:142
13785991310,2cyclictest28693-21cat20:45:262
13785991310,2cyclictest27219-21grep19:35:202
13785991310,2cyclictest25864-21ls19:35:002
13785991310,2cyclictest22679-21grep22:35:212
13785991310,2cyclictest17024-21dpkg23:30:112
13785991310,2cyclictest16572-21sh23:30:002
13785991310,2cyclictest14744-21grep22:30:252
1378399139,3cyclictest920-21tail23:20:161
1378399139,3cyclictest19272-21ssh23:11:431
1378399139,3cyclictest14152-21tr19:10:151
1378399137,5cyclictest25482-21apt-get23:35:151
1378399134,8cyclictest6337-21diskmemload21:25:031
13783991310,2cyclictest30173-21grep22:20:251
13783991310,2cyclictest22503-21chrt23:52:041
13783991310,2cyclictest22406-21cat19:25:201
1377899139,3cyclictest17869-21tr20:25:160
1377899139,3cyclictest12728-21ls00:05:220
1377899139,2cyclictest29603-21cut20:50:140
13778991312,1cyclictest0-21swapper/021:53:430
1377899131,1cyclictest141rcu_preempt23:41:090
13778991310,2cyclictest8090-21wc21:10:200
13778991310,2cyclictest8011-21wc20:05:010
13778991310,2cyclictest8011-21wc20:05:000
13778991310,2cyclictest6321-21cron21:09:590
13778991310,2cyclictest29709-21sed19:40:210
13778991310,2cyclictest17046-21wc23:10:220
1378899129,2cyclictest625-21basename19:50:013
1378899129,2cyclictest6200-21grep20:00:163
1378899129,2cyclictest6200-21grep20:00:163
1378899129,2cyclictest5952-21vmstat21:05:283
1378899129,2cyclictest580-21ssh21:25:153
1378899129,2cyclictest3739-21cat22:25:003
1378899129,2cyclictest30226-21head20:50:173
1378899129,2cyclictest26010-21wc20:40:213
1378899129,2cyclictest2357-21ssh23:40:173
1378899129,2cyclictest19909-21wc20:30:003
1378899129,2cyclictest19858-21basename22:15:123
1378899129,2cyclictest18646-21grep23:50:143
1378899129,2cyclictest10845-21sh00:05:003
1378899128,3cyclictest5823-21ssh21:47:173
1378899128,3cyclictest31772-21ssh21:25:003
1378899121,9cyclictest12245-21cat22:29:553
1378599129,2cyclictest9504-21cat20:05:262
1378599129,2cyclictest7724-21grep21:10:172
1378599129,2cyclictest4519-21wc19:55:222
1378599129,2cyclictest22730-21hostname22:55:132
1378599129,2cyclictest2211-21ssh23:40:162
1378599129,2cyclictest20610-21grep20:30:162
1378599129,2cyclictest1798-21cat21:25:252
1378599129,2cyclictest17420-21wc20:25:012
1378599128,3cyclictest25527-21grep20:40:162
1378599128,3cyclictest22406-21cut00:30:172
13785991212,0cyclictest0-21swapper/223:02:512
13785991210,1cyclictest17100-21ssh23:49:262
1378599120,9cyclictest0-21swapper/219:15:112
1378599120,11cyclictest0-21swapper/222:40:132
1378399129,2cyclictest4194-21uname19:55:191
1378399129,2cyclictest38150irq/131-snd_hda22:12:281
1378399129,2cyclictest27355-21head19:35:221
1378399128,3cyclictest673-21head20:55:231
1378399128,3cyclictest29238-21awk20:50:001
1378399128,3cyclictest2601-21cut21:00:161
1378399127,4cyclictest24913-21ssh22:17:541
1378399125,6cyclictest17016-21grep19:15:161
1378399120,1cyclictest0-21swapper/123:05:141
1378399120,1cyclictest0-21swapper/121:52:541
1377899129,2cyclictest5834-21wc22:25:230
1377899129,2cyclictest23466-21wc00:30:260
1377899129,2cyclictest21326-21ssh22:15:230
1377899129,2cyclictest13598-21cut22:30:160
1377899129,1cyclictest6337-21diskmemload23:56:310
1377899128,3cyclictest21925-21grep19:25:160
1377899128,2cyclictest9403-21munin-plugin-st21:50:000
1377899128,2cyclictest26896-21sed19:35:170
1377899127,3cyclictest5406-21basename20:00:000
1377899121,8cyclictest17627-21head19:15:210
13778991211,1cyclictest161rcuc/021:41:020
1377899120,11cyclictest11210-21awk23:45:200
1378899119,1cyclictest22081-21grep22:35:173
1378899118,2cyclictest9750-21ssh21:11:033
1378899118,2cyclictest30320-21chrt19:41:133
1378899118,2cyclictest29843-21tr22:40:133
1378899118,2cyclictest27415-21tr20:45:143
1378899118,2cyclictest25293-21ssh21:40:093
1378899118,2cyclictest22557-21tr20:35:143
1378899118,2cyclictest19281-21ssh21:35:383
1378899118,2cyclictest15888-21grep20:20:173
1378899118,2cyclictest14295-21grep19:10:143
1378899118,2cyclictest12905-21grep20:15:143
1378899118,2cyclictest12644-21ssh21:51:013
1378899110,10cyclictest9227-21grep21:30:163
1378899110,10cyclictest24464-21head19:30:173
1378599119,1cyclictest4954-21grep21:05:172
1378599119,1cyclictest17862-21grep20:25:152
1378599119,1cyclictest1549-21cron21:00:002
1378599118,2cyclictest4304-21cut00:00:222
1378599118,2cyclictest3968-21awk21:04:592
1378599118,2cyclictest3615-21grep22:05:162
1378599118,2cyclictest31547-21ssh23:38:362
1378599118,2cyclictest31361-21taskset21:24:422
1378599118,2cyclictest28506-21basename00:15:132
1378599118,2cyclictest26894-21aten2_r7power_p22:00:142
1378599118,2cyclictest26894-21aten2_r7power_p22:00:132
1378599118,2cyclictest26641-21cat21:40:202
1378599118,2cyclictest25269-21grep23:15:202
1378599118,2cyclictest2108-21grep19:50:202
1378599118,2cyclictest20491-21grep22:15:142
1378599118,2cyclictest18056-21rm22:51:482
1378599118,2cyclictest11862-21sh21:31:262
1378599118,2cyclictest11732-21ssh23:07:242
1378599118,2cyclictest11446-21cut22:10:132
1378599117,4cyclictest5782-21sh22:45:132
1378599111,1cyclictest15777-21cut23:10:152
1378599110,11cyclictest0-21swapper/221:48:012
1378599110,11cyclictest0-21swapper/200:25:322
1378399118,2cyclictest4923-21taskset21:05:171
1378399118,2cyclictest32546-21ssh00:35:481
1378399118,2cyclictest21331-21grep20:30:221
1378399118,2cyclictest17475-21expr23:30:131
1378399118,2cyclictest1542-21ssh23:40:131
1378399118,2cyclictest13837-21ssh00:05:471
1378399117,3cyclictest29536-21tr20:50:131
1378399116,2cyclictest12410-21taskset20:13:041
1378399116,2cyclictest12410-21taskset20:13:041
1378399113,4cyclictest225-21systemd-journal20:10:011
13783991111,0cyclictest0-21swapper/100:12:241
1378399110,10cyclictest26796-21taskset20:44:191
1378399110,10cyclictest20955-21taskset19:23:081
1377899118,2cyclictest984-21ssh22:41:430
1377899118,2cyclictest9742-21grep20:05:260
1377899118,2cyclictest9067-21ssh00:22:270
1377899118,2cyclictest8922-21tr23:25:160
1377899118,2cyclictest7313-21grep23:05:140
1377899118,2cyclictest32033-21ssh23:00:180
1377899118,2cyclictest31997-21chrt19:45:190
1377899118,2cyclictest28352-21tr20:45:220
1377899118,2cyclictest20606-21ssh23:50:360
1377899118,2cyclictest20394-21sensors_temp19:20:230
1377899118,2cyclictest17746-21ssh21:15:420
1377899118,2cyclictest17498-21wc20:25:000
1377899118,2cyclictest16065-21rm21:34:430
1377899118,2cyclictest13263-21cut00:25:150
1377899110,10cyclictest25623-21ssh22:37:140
1378899108,1cyclictest9839-21chrt20:05:353
1378899108,1cyclictest19377-21expr19:20:153
1378899107,2cyclictest3959-21sh21:04:593
1378899107,2cyclictest32554-21grep20:55:193
1378899107,2cyclictest26872-21expr19:35:163
1378899107,2cyclictest12576-21munin-run20:15:013
1378899107,2cyclictest12576-21munin-run20:15:003
1378599108,1cyclictest30609-21ls00:35:172
1378599108,1cyclictest30120-21fschecks_count20:50:162
1378599108,1cyclictest26983-21aten2_r7power_v23:55:152
1378599108,1cyclictest22572-21ls20:35:142
1378599108,1cyclictest18492-21egrep21:55:132
1378599108,1cyclictest14315-21cat19:10:142
1378599108,1cyclictest11921-21cat20:10:242
1378599108,1cyclictest11921-21cat20:10:242
1378599106,3cyclictest5756-21grep20:00:152
1378599106,3cyclictest5756-21grep20:00:142
1378599103,6cyclictest55850irq/129-enp2s0-21:50:112
1378399108,1cyclictest22621-21expr20:35:141
1378399108,1cyclictest19622-21chrt20:28:531
1378399107,2cyclictest9010-21expr23:25:171
1378399107,2cyclictest32020-21tr19:45:201
1377899108,1cyclictest2718-21taskset19:51:360
1377899108,1cyclictest26620-21ssh22:00:050
1377899108,1cyclictest26620-21ssh22:00:050
1377899108,1cyclictest24757-21apt-get20:40:130
1377899108,1cyclictest23276-21sed22:55:150
1377899108,1cyclictest21968-21chrt20:33:310
1377899108,1cyclictest18898-21sed21:55:140
1377899108,1cyclictest13664-21basename20:15:190
1377899107,2cyclictest32275-21ssh21:25:050
1377899107,2cyclictest23146-21head20:35:170
1377899107,2cyclictest11790-21which20:10:210
1377899107,2cyclictest11790-21which20:10:210
1377899107,2cyclictest1089-21grep20:55:270
1377899100,9cyclictest18122sleep021:00:080
137859997,1cyclictest19099-21cut19:20:152
137859996,2cyclictest13461-21uniq20:15:162
137839997,1cyclictest903-21cut19:50:121
137839997,1cyclictest28888-21expr19:40:141
137839997,1cyclictest13576-21expr20:15:171
137789997,1cyclictest23714-21taskset19:30:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional