You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 02:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Jul 02, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
279229552,39sleep20-21swapper/219:07:232
287429380,9sleep00-21swapper/019:08:260
267629063,9sleep30-21swapper/319:05:523
85728464,16sleep10-21swapper/119:05:101
238902580,0sleep30-21swapper/323:02:253
308999270,27cyclictest0-21swapper/221:28:222
308999260,26cyclictest0-21swapper/221:24:362
308499251,2cyclictest0-21swapper/022:42:270
3089992423,1cyclictest0-21swapper/223:23:432
308999240,24cyclictest0-21swapper/200:08:472
3089992322,1cyclictest0-21swapper/221:11:042
308999230,22cyclictest0-21swapper/221:46:222
308699231,3cyclictest0-21swapper/100:34:061
308499221,3cyclictest0-21swapper/020:10:130
308999211,20cyclictest0-21swapper/200:19:202
308999210,21cyclictest0-21swapper/200:25:542
308699210,2cyclictest0-21swapper/121:55:241
308499211,15cyclictest24291-21taskset21:45:200
308499210,1cyclictest0-21swapper/021:20:320
112112210,1sleep111215-21tr22:55:141
308999200,19cyclictest0-21swapper/222:28:122
308999200,19cyclictest0-21swapper/220:10:232
308999200,17cyclictest0-21swapper/219:10:132
3086992016,3cyclictest0-21swapper/121:20:411
308699201,1cyclictest28307-21diskmemload22:02:251
3089991912,6cyclictest24209-21apache_volume22:05:112
308999190,19cyclictest0-21swapper/222:00:332
308999180,18cyclictest0-21swapper/223:27:082
308999180,17cyclictest0-21swapper/222:36:382
3086991816,1cyclictest38150irq/131-snd_hda00:10:091
308699180,1cyclictest0-21swapper/123:05:491
308699180,18cyclictest0-21swapper/100:35:231
308499185,10cyclictest5213-21apt-get23:30:130
3084991817,1cyclictest0-21swapper/019:52:330
3084991815,2cyclictest13-21ksoftirqd/000:18:020
3091991712,4cyclictest6798-21grep21:15:273
308999171,15cyclictest27045-21irqstats21:05:172
308999170,17cyclictest0-21swapper/223:00:352
308999170,17cyclictest0-21swapper/222:17:122
308999170,16cyclictest0-21swapper/222:53:492
308699177,9cyclictest0-21swapper/123:15:121
308699170,1cyclictest0-21swapper/122:46:271
308699170,1cyclictest0-21swapper/121:48:461
308499177,9cyclictest27477-21apache222:45:230
3084991715,2cyclictest13-21ksoftirqd/023:04:380
3084991710,4cyclictest13-21ksoftirqd/022:55:010
308499170,11cyclictest0-21swapper/019:15:140
308499170,0cyclictest0-21swapper/023:18:380
309199160,3cyclictest0-21swapper/322:00:253
308999160,15cyclictest0-21swapper/222:42:262
3086991613,2cyclictest38150irq/131-snd_hda22:10:131
3086991611,4cyclictest1458-21tune2fs00:25:171
308699161,0cyclictest0-21swapper/121:14:371
308699160,15cyclictest28307-21diskmemload22:42:111
308699160,15cyclictest0-21swapper/120:05:121
308699160,13cyclictest0-21swapper/122:39:481
309199152,9cyclictest19411-21fschecks_time20:50:153
308699151,13cyclictest5807-21ssh00:08:511
308699150,12cyclictest0-21swapper/121:33:161
309199141,3cyclictest11175-21vmstat00:30:273
309199141,1cyclictest0-21swapper/300:24:183
3091991410,3cyclictest22475-21cron21:45:003
3091991410,3cyclictest11821-21ssh22:55:173
309199140,13cyclictest29620-21tail21:10:163
3089991411,3cyclictest0-21swapper/221:15:322
308999140,14cyclictest0-21swapper/222:22:232
308999140,14cyclictest0-21swapper/219:18:132
308999140,13cyclictest0-21swapper/221:57:072
308699146,6cyclictest28307-21diskmemload21:25:081
3086991413,1cyclictest3076-21cyclictest22:54:211
3086991412,2cyclictest0-21swapper/122:20:261
3086991411,2cyclictest0-21swapper/100:00:171
3086991410,3cyclictest32200-21turbostat.cron22:09:591
3086991410,3cyclictest28486-21cron23:25:001
3086991410,2cyclictest18901-21grep20:50:141
308699140,11cyclictest0-21swapper/122:30:121
3084991413,1cyclictest28307-21diskmemload23:59:050
3084991410,1cyclictest28307-21diskmemload21:50:380
309199139,2cyclictest30588-21tr21:30:153
309199134,8cyclictest28307-21diskmemload22:32:373
3091991310,2cyclictest5615-21wc19:15:013
3091991310,2cyclictest32622-21sed00:05:223
3091991310,2cyclictest28624-21cat20:00:263
3091991310,2cyclictest26337-21grep19:55:253
3091991310,2cyclictest25856-21cron21:05:013
3091991310,2cyclictest22380-21sort23:20:273
3091991310,2cyclictest14504-21grep20:40:173
308999137,6cyclictest27533-21apache222:10:472
3089991310,2cyclictest4962-21grep20:20:152
3089991310,2cyclictest23255-21dump-pmu-power21:00:012
3089991310,2cyclictest19429-21tune2fs20:50:162
3089991310,2cyclictest17367-21cat19:40:002
308999130,1cyclictest0-21swapper/223:55:292
308999130,13cyclictest0-21swapper/200:35:382
308999130,12cyclictest0-21swapper/221:52:082
308699139,3cyclictest5463-21ssh21:15:161
308699139,3cyclictest21235-21wc19:45:211
308699139,2cyclictest17354-21sleep19:40:001
308699138,4cyclictest19623-21ssh23:38:421
3086991310,2cyclictest14100-21tr21:40:001
3086991310,2cyclictest12561-21cron19:30:001
3086991310,2cyclictest10657-21grep20:30:261
308499136,6cyclictest4030-21awk20:20:000
308499135,3cyclictest225-21systemd-journal20:41:340
308499131,8cyclictest0-21swapper/021:55:120
308499130,12cyclictest27259-21diskmemload23:43:050
308499130,12cyclictest0-21swapper/000:32:410
308499130,0cyclictest0-21swapper/023:25:100
309199129,2cyclictest9391-21cut22:15:153
309199129,2cyclictest9282-21tr20:30:143
309199129,2cyclictest8149-21tail21:35:283
309199129,2cyclictest6460-21ssh23:30:203
309199129,2cyclictest5729-21grep19:15:143
309199129,2cyclictest27889-21tr22:45:213
309199129,2cyclictest24379-21tail00:00:193
309199129,2cyclictest21172-21grep20:55:143
309199129,2cyclictest1890-21wc00:25:193
309199129,2cyclictest17118-21grep00:15:163
309199129,2cyclictest16699-21sed23:55:273
309199129,2cyclictest13904-21ssh23:15:233
309199129,2cyclictest12569-21sh19:30:003
309199128,3cyclictest24558-21ssh21:45:223
309199126,2cyclictest225-21systemd-journal21:25:003
309199123,8cyclictest13390-21/usr/sbin/munin23:35:153
309199121,2cyclictest141rcu_preempt21:55:143
3091991211,1cyclictest0-21swapper/323:29:143
309199120,6cyclictest26362-21cpuspeed_turbos21:05:133
309199120,11cyclictest24605-21expr22:05:143
308999129,2cyclictest6412-21cut21:35:162
308999129,2cyclictest22086-21grep23:40:122
308999129,2cyclictest12978-21cat20:35:262
308999129,2cyclictest12389-21tr23:15:122
308999128,3cyclictest15903-21cron20:44:592
308999125,7cyclictest23112-21munin-node21:32:342
308999125,7cyclictest0-21swapper/219:33:292
308999122,6cyclictest225-21systemd-journal22:50:012
308999121,2cyclictest27790-21apache_volume23:05:132
308999120,5cyclictest225-21systemd-journal20:19:592
308999120,1cyclictest0-21swapper/219:25:142
308999120,12cyclictest0-21swapper/200:12:432
308999120,10cyclictest0-21swapper/200:00:132
308699129,2cyclictest8326-21tail20:25:261
308699129,2cyclictest776-21cat21:50:271
308699129,2cyclictest66472sleep119:15:201
308699129,2cyclictest28486-21wc21:10:011
308699129,2cyclictest15456-21sed21:40:161
308699127,2cyclictest8153-21ssh23:31:101
308699125,4cyclictest225-21systemd-journal19:55:171
308699120,12cyclictest0-21swapper/100:18:211
308499129,2cyclictest31248-21grep20:05:250
308499129,2cyclictest30412-21ssh23:45:130
308499129,2cyclictest25400-21basename00:40:010
308499129,2cyclictest21939-21ls23:20:220
308499129,2cyclictest21294-21cron23:40:000
308499129,2cyclictest11692-21tr20:35:140
3084991210,2cyclictest13-21ksoftirqd/021:30:060
3084991210,1cyclictest5056-21sed20:20:160
3084991210,1cyclictest26874-21cut21:05:150
3084991210,1cyclictest25066-21cut22:05:160
3084991210,1cyclictest10597-21grep22:35:170
3084991210,1cyclictest0-21swapper/022:11:250
309199119,1cyclictest28798-21interrupts23:05:193
309199119,1cyclictest2665-21head20:15:183
309199118,2cyclictest966-21ssh23:46:163
309199118,2cyclictest8786-21munin-run20:29:593
309199118,2cyclictest8013-21aten2_r7power_c00:10:153
309199118,2cyclictest7361-21ssh22:52:363
309199118,2cyclictest6685-21ssh23:11:103
309199118,2cyclictest6555-21tr23:50:133
309199118,2cyclictest494-21ssh22:29:443
309199118,2cyclictest4521-21grep20:20:133
309199118,2cyclictest2556-21ssh22:10:393
309199118,2cyclictest24816-21ssh23:41:123
309199118,2cyclictest22150-21cat21:25:143
309199118,2cyclictest22073-21cron19:50:003
309199118,2cyclictest20434-21ssh22:40:383
309199118,2cyclictest17815-21sed20:45:263
309199118,2cyclictest17743-21cut19:40:143
309199118,2cyclictest17271-21ssh00:35:023
309199118,2cyclictest16009-21taskset19:35:183
309199118,2cyclictest12683-21taskset22:36:103
309199118,2cyclictest1153-21chrt21:50:313
309199117,3cyclictest9315-21head19:20:193
308999119,2cyclictest0-21swapper/223:47:032
308999119,1cyclictest5109-21grep23:10:212
308999119,1cyclictest13545-21wget23:35:152
308999118,2cyclictest8049-21sh23:31:062
308999118,2cyclictest6436-21ssh23:50:132
308999118,2cyclictest25616-21sed00:20:182
308999118,2cyclictest23027-21grep19:50:182
308999118,2cyclictest19703-21cron19:45:002
308999118,2cyclictest18296-21awk20:50:002
3089991110,1cyclictest0-21swapper/200:30:302
308999110,10cyclictest0-21swapper/221:40:242
308699118,2cyclictest30278-21ssh23:45:111
308699118,2cyclictest29730-21ssh22:27:091
308699118,2cyclictest25991-21tail00:20:201
308699118,2cyclictest24251-21ssh23:40:441
308699118,2cyclictest23549-21sed19:50:231
308699118,2cyclictest22392-21wc20:55:221
308699118,2cyclictest1900-21cut20:15:151
308699118,2cyclictest17575-21cat19:40:111
308699118,2cyclictest16623-21grep23:55:251
308699118,2cyclictest12688-21ssh23:53:501
308699117,3cyclictest4036-21head19:10:161
308699117,3cyclictest21649-21ssh23:00:411
308699116,2cyclictest31070-21ssh23:25:301
308699113,5cyclictest7205-21sshd23:11:341
308699111,6cyclictest225-21systemd-journal20:05:001
308499119,1cyclictest22773-21grep20:55:270
308499119,1cyclictest17379-21grep22:20:150
308499119,1cyclictest12100-21grep19:25:260
308499118,2cyclictest998-21sed00:05:290
308499118,2cyclictest9301-21tr20:30:130
308499118,2cyclictest8140-21grep21:35:280
308499118,2cyclictest6305-21sh22:32:420
308499118,2cyclictest6206-21ssh23:50:090
308499118,2cyclictest5798-21ssh23:10:280
308499118,2cyclictest31765-21chrt23:06:580
308499118,2cyclictest29917-21sed21:10:180
308499118,2cyclictest27352-21cut20:00:150
308499118,2cyclictest26231-21ssh22:25:170
308499118,2cyclictest26185-21wc00:20:220
308499118,2cyclictest2538-21cat00:25:260
308499118,2cyclictest25329-21sort19:55:160
308499118,2cyclictest24415-21cut21:00:160
308499118,2cyclictest23810-21uniq00:00:160
308499118,2cyclictest21777-21apache_accesses21:25:120
308499118,2cyclictest20220-21tr19:45:140
308499118,2cyclictest20158-21python20:50:240
308499118,2cyclictest18279-21cron20:50:000
308499118,2cyclictest15639-21sh22:57:320
308499118,2cyclictest12920-21ssh00:12:440
308499118,2cyclictest10156-21chrt19:24:360
308499116,4cyclictest28307-21diskmemload22:00:250
308499115,1cyclictest9912-21ntp_kernel_err22:15:190
308499110,10cyclictest7439-21cut20:25:170
308499110,10cyclictest5465-21df21:15:170
309199108,1cyclictest24208-21taskset19:51:533
309199107,2cyclictest31348-21taskset20:06:033
309199107,2cyclictest16791-21ssh22:20:013
309199107,2cyclictest14652-21chrt19:31:233
308999108,2cyclictest9893-21taskset19:21:132
308999108,1cyclictest7136-21cat20:25:152
308999108,1cyclictest25460-21sed21:00:262
308999108,1cyclictest11813-21head22:55:172
308999107,2cyclictest3873-21ssh22:30:482
308999107,2cyclictest20620-21tune2fs19:45:172
308699108,1cyclictest8529-21expr19:20:141
308699108,1cyclictest32005-21awk20:10:141
308699108,1cyclictest23808-21cut21:00:141
308699108,1cyclictest14017-21wc20:40:141
308699108,1cyclictest13043-21cat19:30:131
308699107,2cyclictest8925-21cut22:15:141
308699107,2cyclictest16491-21expr20:45:151
308699107,2cyclictest11724-21seq20:35:141
308699100,9cyclictest4662-21cat20:20:131
308499108,1cyclictest32822sleep019:10:100
308499107,2cyclictest18314-21tail19:40:170
308499107,2cyclictest15828-21tune2fs19:35:160
308499107,2cyclictest14251-21grep19:30:240
308499104,5cyclictest27531-21apache221:41:120
30919997,1cyclictest31887-21chrt20:10:033
30919997,1cyclictest11468-21cut20:35:143
30899996,2cyclictest9821-21head20:30:162
30899996,2cyclictest27269-21egrep20:00:152
30899991,7cyclictest24732-21apt-get19:55:122
30899990,3cyclictest30478-21sshd20:05:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional