You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-26 - 14:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Mar 26, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
133399274,0cyclictest62550irq/127-eno123:23:460
1343992523,1cyclictest10785-21df21:57:573
134399250,24cyclictest0-21swapper/323:34:363
1336992522,2cyclictest18076-21sh00:13:511
133699250,24cyclictest28198-21diskmemload23:38:021
133699250,24cyclictest15862-21sh21:29:501
133399250,24cyclictest28198-21diskmemload21:54:340
1343992422,1cyclictest0-21swapper/321:27:593
134399240,24cyclictest28198-21diskmemload23:28:363
134399240,23cyclictest0-21swapper/322:26:453
1341992423,0cyclictest6018-21apt-get00:02:582
134199240,23cyclictest0-21swapper/223:53:592
134199240,23cyclictest0-21swapper/221:58:002
134199240,1cyclictest28198-21diskmemload22:25:252
134199240,1cyclictest28198-21diskmemload00:29:132
134199240,1cyclictest151rcu_preempt22:19:592
134199240,0cyclictest0-21swapper/200:05:452
1336992423,1cyclictest32079-21ssh21:13:131
1336992423,1cyclictest28198-21diskmemload22:17:071
1336992422,1cyclictest0-21swapper/123:19:061
133699240,24cyclictest28198-21diskmemload23:27:211
133699240,23cyclictest0-21swapper/122:06:111
133699240,0cyclictest0-21swapper/122:46:131
133699240,0cyclictest0-21swapper/100:04:191
1333992423,1cyclictest62550irq/127-eno121:38:590
1333992423,1cyclictest23155-21apt-get23:47:570
1333992421,2cyclictest19384-21cut19:45:010
133399240,23cyclictest32141-21sh22:50:330
133399240,23cyclictest28198-21diskmemload00:37:160
133399240,23cyclictest10891-21sh22:29:510
133399240,23cyclictest0-21swapper/022:12:500
133399240,0cyclictest28198-21diskmemload23:58:430
133399240,0cyclictest28198-21diskmemload21:26:560
133399240,0cyclictest0-21swapper/000:15:560
1343992323,0cyclictest0-21swapper/323:50:493
1343992323,0cyclictest0-21swapper/322:36:553
1343992323,0cyclictest0-21swapper/321:30:493
1343992323,0cyclictest0-21swapper/300:13:393
1343992321,1cyclictest1529-21ntp_states21:48:043
1343992321,1cyclictest0-21swapper/322:49:053
1343992321,1cyclictest0-21swapper/322:16:233
1343992321,1cyclictest0-21swapper/321:20:013
134399230,2cyclictest30437-21sh23:22:113
134399230,23cyclictest28198-21diskmemload22:56:573
134399230,23cyclictest0-21swapper/322:11:243
134399230,23cyclictest0-21swapper/322:05:383
134399230,23cyclictest0-21swapper/321:35:023
134399230,22cyclictest28198-21diskmemload23:15:243
134399230,22cyclictest28198-21diskmemload22:30:173
134399230,22cyclictest23965-21grep23:15:013
134399230,22cyclictest15993-21ssh23:39:333
134399230,1cyclictest0-21swapper/323:46:243
134399230,0cyclictest28198-21diskmemload00:26:383
134399230,0cyclictest28198-21diskmemload00:17:383
134399230,0cyclictest0-21swapper/322:04:233
1341992323,0cyclictest0-21swapper/223:25:242
1341992321,1cyclictest23398-21cron21:00:002
1341992321,1cyclictest13944-21ssh22:00:352
1341992321,1cyclictest0-21swapper/223:48:052
1341992321,1cyclictest0-21swapper/221:15:152
1341992320,2cyclictest18195-21ssh22:37:432
1341992320,2cyclictest0-21swapper/221:37:572
134199230,23cyclictest0-21swapper/223:02:072
134199230,23cyclictest0-21swapper/222:55:332
134199230,23cyclictest0-21swapper/222:21:402
134199230,23cyclictest0-21swapper/221:32:452
134199230,23cyclictest0-21swapper/200:34:242
134199230,22cyclictest28198-21diskmemload23:20:452
134199230,22cyclictest28198-21diskmemload00:36:492
134199230,22cyclictest14141-21cron19:35:002
134199230,22cyclictest12735-21ssh22:31:502
134199230,22cyclictest0-21swapper/223:10:422
134199230,1cyclictest28198-21diskmemload23:34:222
134199230,1cyclictest0-21swapper/223:41:312
134199230,1cyclictest0-21swapper/200:15:472
134199230,0cyclictest24711-21ssh23:15:512
134199230,0cyclictest0-21swapper/222:51:152
134199230,0cyclictest0-21swapper/221:50:102
134199230,0cyclictest0-21swapper/221:41:162
134199230,0cyclictest0-21swapper/221:29:222
1336992323,0cyclictest28198-21diskmemload22:27:521
1336992323,0cyclictest0-21swapper/100:26:221
1336992322,1cyclictest0-21swapper/122:50:491
1336992321,1cyclictest28198-21diskmemload23:48:411
1336992321,1cyclictest18250-21apt-get23:42:571
1336992321,1cyclictest0-21swapper/123:50:011
1336992321,1cyclictest0-21swapper/123:00:441
1336992321,1cyclictest0-21swapper/119:32:561
133699230,2cyclictest20790-21ssh22:39:121
133699230,23cyclictest0-21swapper/122:23:501
133699230,23cyclictest0-21swapper/121:45:051
133699230,22cyclictest10880-21ssh21:57:581
133699230,22cyclictest0-21swapper/100:05:571
133699230,1cyclictest23055-21ssh23:13:331
133699230,0cyclictest28198-21diskmemload23:20:191
133699230,0cyclictest28198-21diskmemload22:10:211
1333992323,0cyclictest0-21swapper/023:28:390
1333992323,0cyclictest0-21swapper/022:20:330
1333992323,0cyclictest0-21swapper/022:15:090
1333992323,0cyclictest0-21swapper/021:56:300
1333992323,0cyclictest0-21swapper/021:30:230
1333992321,1cyclictest0-21swapper/022:35:210
1333992321,1cyclictest0-21swapper/000:21:100
133399230,2cyclictest6809-21ssh23:30:310
133399230,2cyclictest19535-21ssh23:10:250
133399230,2cyclictest16800-21ssh23:40:330
133399230,23cyclictest0-21swapper/023:15:310
133399230,23cyclictest0-21swapper/023:00:030
133399230,23cyclictest0-21swapper/000:25:530
133399230,22cyclictest27574-21sh21:42:240
133399230,22cyclictest23402-21ls21:00:000
133399230,22cyclictest22546-21ssh22:41:390
133399230,22cyclictest0-21swapper/022:58:030
133399230,1cyclictest0-21swapper/021:20:350
133399230,0cyclictest28198-21diskmemload23:39:430
133399230,0cyclictest0-21swapper/023:05:580
1343992222,0cyclictest0-21swapper/323:02:143
1343992222,0cyclictest0-21swapper/321:10:083
1343992222,0cyclictest0-21swapper/300:02:153
1343992221,0cyclictest0-21swapper/321:40:423
1343992221,0cyclictest0-21swapper/319:15:023
1343992220,1cyclictest31686-21sh23:55:043
1343992220,1cyclictest0-21swapper/322:40:103
134399220,2cyclictest6779-21ssh21:20:513
134399220,21cyclictest313-21systemd-journal21:05:003
134399220,21cyclictest28198-21diskmemload21:50:203
134399220,21cyclictest0-21swapper/323:40:113
134399220,21cyclictest0-21swapper/321:07:573
134399220,21cyclictest0-21swapper/320:22:573
134399220,21cyclictest0-21swapper/319:55:013
134399220,21cyclictest0-21swapper/300:30:343
134399220,21cyclictest0-21swapper/300:05:353
134399220,1cyclictest0-21swapper/322:50:013
134399220,1cyclictest0-21swapper/300:22:553
1341992222,0cyclictest0-21swapper/222:43:172
1341992222,0cyclictest0-21swapper/222:10:572
1341992220,2cyclictest29707-21sh22:48:052
1341992220,1cyclictest606-21dbus-daemon21:10:002
1341992219,0cyclictest0-21swapper/223:39:452
134199220,21cyclictest15047-21ssh23:06:152
134199220,21cyclictest0-21swapper/223:58:042
134199220,21cyclictest0-21swapper/219:22:572
134199220,1cyclictest22077-21ssh22:08:072
134199220,1cyclictest0-21swapper/221:45:342
134199220,0cyclictest0-21swapper/200:21:332
1336992222,0cyclictest0-21swapper/123:55:361
1336992222,0cyclictest0-21swapper/122:56:081
1336992222,0cyclictest0-21swapper/122:30:351
1336992222,0cyclictest0-21swapper/100:32:371
1336992221,1cyclictest0-21swapper/121:21:091
1336992220,2cyclictest22025-21ssh21:36:261
133699220,22cyclictest0-21swapper/121:41:541
133699220,21cyclictest28198-21diskmemload21:16:081
133699220,21cyclictest0-21swapper/121:53:381
133699220,21cyclictest0-21swapper/120:57:561
133699220,21cyclictest0-21swapper/100:20:201
133699220,1cyclictest7324-21ssh00:35:451
133699220,1cyclictest0-21swapper/123:34:181
133699220,1cyclictest0-21swapper/122:00:391
133699220,1cyclictest0-21swapper/100:15:011
133699220,0cyclictest28198-21diskmemload21:30:011
1333992222,0cyclictest0-21swapper/020:22:580
1333992221,1cyclictest14149-21ssh00:10:080
1333992220,2cyclictest75-21kswapd022:30:030
1333992220,1cyclictest16976-21munin-run19:40:000
1333992220,1cyclictest13846-21perf00:10:000
1333992220,1cyclictest0-21swapper/021:15:180
133399220,2cyclictest28220-21perf21:10:000
133399220,22cyclictest28198-21diskmemload00:02:090
133399220,22cyclictest0-21swapper/022:46:110
133399220,21cyclictest28739-21ssh21:10:320
133399220,21cyclictest0-21swapper/022:05:030
133399220,1cyclictest0-21swapper/023:51:220
133399220,1cyclictest0-21swapper/022:00:290
133399220,1cyclictest0-21swapper/000:32:530
133399220,0cyclictest0-21swapper/020:02:570
1343992120,1cyclictest15034-21apt-get19:37:563
1343992120,0cyclictest14309-21apt-get20:42:573
1343992120,0cyclictest12253-21apt-get19:32:563
134399210,20cyclictest8908-21cron19:25:003
134399210,0cyclictest0-21swapper/320:27:573
1341992121,0cyclictest0-21swapper/221:20:062
1341992121,0cyclictest0-21swapper/221:10:502
1341992120,0cyclictest0-21swapper/220:47:582
1341992119,1cyclictest7045-21apt-get20:27:562
134199210,1cyclictest14864-21ssh00:11:252
1336992120,1cyclictest31844-21apt-get20:12:571
1336992120,1cyclictest16704-21apt-get20:47:581
133699210,20cyclictest0-21swapper/119:37:561
133699210,20cyclictest0-21swapper/119:22:571
133699210,0cyclictest0-21swapper/123:06:021
133699210,0cyclictest0-21swapper/120:17:571
1333992121,0cyclictest28198-21diskmemload21:45:090
1333992120,1cyclictest0-21swapper/020:35:010
1333992120,0cyclictest0-21swapper/020:37:550
1333992119,1cyclictest606-21dbus-daemon20:00:010
133399210,20cyclictest0-21swapper/020:47:570
133399210,20cyclictest0-21swapper/020:12:570
1343992020,0cyclictest29452-21apt-get20:07:583
1343992019,1cyclictest9404-21apt-get19:27:583
1343992019,0cyclictest1797-21apt-get19:12:573
1343992017,2cyclictest3772-21awk20:20:003
134399200,0cyclictest17454-21apt-get19:42:563
134399200,0cyclictest0-21swapper/319:47:573
1341992020,0cyclictest0-21swapper/219:52:572
134199200,0cyclictest0-21swapper/219:17:562
1336992020,0cyclictest0-21swapper/120:32:561
1336992020,0cyclictest0-21swapper/120:02:571
1333992020,0cyclictest0-21swapper/019:52:570
1333992018,1cyclictest3748-21ls20:20:000
133399200,0cyclictest0-21swapper/021:02:560
1343991919,0cyclictest24640-21apt-get19:57:573
1343991919,0cyclictest0-21swapper/320:37:553
1343991917,1cyclictest23380-21cron21:00:003
1341991919,0cyclictest0-21swapper/220:17:572
1341991919,0cyclictest0-21swapper/219:57:572
1341991919,0cyclictest0-21swapper/219:27:572
1341991919,0cyclictest0-21swapper/219:12:572
134199190,18cyclictest16928-21sendmail19:40:002
1336991919,0cyclictest0-21swapper/120:42:571
1336991918,0cyclictest4652-21apt-get20:22:571
1333991919,0cyclictest0-21swapper/020:52:560
1333991918,1cyclictest7010-21apt-get19:22:570
1343991816,1cyclictest0-21swapper/320:12:573
134399180,17cyclictest9444-21apt-get20:32:563
1341991818,0cyclictest0-21swapper/220:22:572
1341991816,1cyclictest1338-21munin-run20:15:002
1336991818,0cyclictest0-21swapper/119:12:571
133399180,1cyclictest14277-21/usr/sbin/munin20:43:030
1341991716,0cyclictest27057-21apt-get20:02:572
1341991716,0cyclictest0-21swapper/219:47:572
1336991717,0cyclictest11891-21apt-get20:37:551
1336991716,0cyclictest0-21swapper/120:52:561
1336991715,1cyclictest783-21snmpd19:25:181
1333991717,0cyclictest0-21swapper/019:12:570
1343991615,0cyclictest0-21swapper/320:47:573
1343991614,1cyclictest313-21systemd-journal20:05:013
1341991616,0cyclictest0-21swapper/220:42:572
1341991614,1cyclictest13810-21latency_hist20:40:012
1341991613,2cyclictest21069-21latency_hist20:55:012
1336991616,0cyclictest0-21swapper/119:57:561
1333991616,0cyclictest0-21swapper/020:27:560
1333991616,0cyclictest0-21swapper/019:17:560
133399160,1cyclictest14180-21munin-run19:35:000
1341991515,0cyclictest0-21swapper/219:42:562
1341991513,1cyclictest23911-21apt-get21:02:562
134199150,1cyclictest0-21swapper/220:08:062
1336991515,0cyclictest0-21swapper/119:15:021
134399140,14cyclictest0-21swapper/320:50:113
1341991414,0cyclictest0-21swapper/220:32:422
133699140,14cyclictest0-21swapper/121:00:161
133699140,14cyclictest0-21swapper/119:45:021
133699140,0cyclictest0-21swapper/120:05:081
133399140,14cyclictest0-21swapper/020:06:220
133399140,14cyclictest0-21swapper/019:45:120
133399140,14cyclictest0-21swapper/019:26:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional