You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 05:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Sep 17, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2106399514,26cyclictest35966irq/58-eth022:10:133
2104799516,39cyclictest11967-21ssh23:55:131
21063995029,12cyclictest35966irq/58-eth021:20:123
21036995037,10cyclictest29726-21apt-get21:00:130
21036994738,7cyclictest5709-21apt-get20:35:120
21063994625,15cyclictest0-21swapper/321:50:133
21063994625,10cyclictest35966irq/58-eth022:25:123
2104799452,38cyclictest22829-21ssh23:25:141
21036994417,10cyclictest30797-21ssh21:21:330
21055994315,19cyclictest13852-21ssh23:20:122
21036994318,11cyclictest9088-21ssh21:45:270
21036994317,8cyclictest9404-21ssh00:29:460
21036994317,16cyclictest16041-21ssh21:31:080
21036994316,8cyclictest24887-21ssh22:30:230
21036994316,10cyclictest9533-21ssh22:59:200
21036994316,10cyclictest9533-21ssh22:59:200
21036994316,10cyclictest26673-21ssh21:38:100
21036994316,10cyclictest15871-21ssh22:08:100
21036994316,10cyclictest15871-21ssh22:08:100
21036994316,10cyclictest14334-21ssh23:36:540
21036994313,8cyclictest26539-21ssh23:45:170
21036994313,11cyclictest24917-21cut21:55:200
21055994231,8cyclictest32023-21apt-get19:55:122
21055994221,15cyclictest15921-21ssh22:45:132
21055994214,21cyclictest0-21swapper/223:45:132
2104799424,17cyclictest21-21ksoftirqd/120:10:121
2104799423,30cyclictest10504-21ssh23:00:131
21036994230,8cyclictest15915-21apt-get22:45:140
21036994218,8cyclictest10596-21irqrtprio00:30:200
21036994217,9cyclictest3152-21cat22:00:210
21036994217,8cyclictest7718-21ssh21:26:560
21036994217,10cyclictest12890-21rm00:12:160
21036994216,10cyclictest3744-21ssh23:14:060
21036994216,10cyclictest12680-21ssh23:19:040
21036994213,11cyclictest7479-21ssh23:34:290
21036994213,11cyclictest7479-21ssh23:34:290
21036994116,9cyclictest30398-21ssh22:51:110
21036994116,9cyclictest2088-21ssh22:18:590
21036994116,11cyclictest15274-21ssh21:14:380
21036994115,10cyclictest32716-21ssh00:24:350
21036994113,9cyclictest23626-21ssh23:05:290
21036994033,4cyclictest16014-21apt-get21:15:150
21036994033,3cyclictest2088-21apt-get21:05:150
21036994016,7cyclictest22005-21ssh23:24:320
21036994015,9cyclictest21705-21ssh00:17:030
21036994015,16cyclictest6666-21diskmemload23:57:560
21036994015,10cyclictest9272-21sort20:05:130
2104799392,26cyclictest6914-21ssh22:40:111
2104799392,25cyclictest26329-21ssh23:45:141
21036993933,3cyclictest12816-21apt-get19:35:120
21036993916,8cyclictest2880-21ssh21:42:420
21036993916,7cyclictest6666-21diskmemload22:35:030
21036993916,7cyclictest18849-21grep21:50:300
21036993916,10cyclictest27638-21rm00:02:400
21036993914,9cyclictest4198-21ssh00:07:350
21036993914,10cyclictest29413-21missed_timers19:15:230
21036993914,10cyclictest10231-21rm22:22:550
21036993912,8cyclictest12576-21sshd19:34:460
21036993911,9cyclictest8769-21ssh23:51:390
21063993815,14cyclictest35966irq/58-eth020:10:123
21055993810,10cyclictest26153-21rm21:37:332
2104799382,23cyclictest24568-21ssh00:20:131
21036993817,16cyclictest0-21swapper/020:25:130
21036993816,9cyclictest26741-21ssh00:38:540
21036993816,13cyclictest6666-21diskmemload22:10:030
21036993815,9cyclictest25128-21idleruntime20:55:180
21036993814,8cyclictest6666-21diskmemload22:29:470
21036993814,8cyclictest24017-21cut23:25:200
21036993810,10cyclictest11709-21cat22:40:290
21055993726,3cyclictest26873-21apt-get22:15:172
21036993715,8cyclictest15545-21rm23:00:410
21036993714,9cyclictest23566-21sh23:42:130
21055993626,7cyclictest10671-21apt-get21:30:122
21055993617,16cyclictest0-21swapper/223:05:122
2104799361,28cyclictest30384-21sh22:35:131
2103699369,9cyclictest664-21ntpd20:00:240
2103699369,11cyclictest24853-21cut19:45:210
2105599357,19cyclictest0-21swapper/223:27:212
2105599353,15cyclictest31965-21ssh23:30:152
2105599353,15cyclictest31965-21ssh23:30:152
21055993529,3cyclictest18893-21apt-get20:15:152
21055993523,9cyclictest23123-21ssh21:55:112
21055993515,16cyclictest0-21swapper/220:45:132
21055993515,15cyclictest0-21swapper/220:55:142
21036993516,4cyclictest22291-21ntp_192.168.11420:15:270
2105599348,10cyclictest26876-21ssh00:39:042
2105599344,20cyclictest0-21swapper/200:17:552
21055993420,12cyclictest27346irq/40-dwc2_hso20:25:132
21055993418,13cyclictest0-21swapper/221:50:112
21055993412,18cyclictest0-21swapper/222:05:122
21055993412,18cyclictest0-21swapper/222:05:122
21055993412,17cyclictest0-21swapper/221:15:152
21036993424,8cyclictest15142-21sed20:45:020
21036993422,7cyclictest21249-21apt-get19:10:110
21036993418,12cyclictest867-21apt-get20:30:120
21036993414,7cyclictest4310-21cat20:00:020
21036993410,10cyclictest31138-21cat19:20:170
2106399331,18cyclictest35966irq/58-eth021:30:123
2105599338,12cyclictest12766-21ssh23:19:112
2105599338,11cyclictest13325-21ssh00:12:482
2105599337,10cyclictest0-21swapper/223:40:062
2105599336,11cyclictest930-21snmpd22:10:382
2105599335,18cyclictest0-21swapper/222:54:212
2105599334,18cyclictest6724-21ps23:50:262
2105599334,12cyclictest0-21swapper/221:49:102
21055993317,14cyclictest0-21swapper/200:20:112
21055993312,12cyclictest0-21swapper/221:10:142
21036993322,8cyclictest3249-21apt-get19:25:110
2106399324,25cyclictest35966irq/58-eth022:30:123
2106399322,18cyclictest35966irq/58-eth021:40:143
2106399321,19cyclictest35966irq/58-eth000:00:143
2105599327,8cyclictest14204-21ssh23:36:442
2105599325,11cyclictest0-21swapper/223:11:392
2105599323,8cyclictest19537-21sh22:28:192
2104799321,23cyclictest10020-21ssh00:30:141
21036993215,8cyclictest18668-21munin-plugin-st20:15:020
21036993214,6cyclictest21042-21/usr/sbin/munin19:40:230
2105599318,11cyclictest2001-21ssh21:41:372
2105599316,9cyclictest0-21swapper/200:26:122
2105599316,11cyclictest0-21swapper/222:04:142
2105599315,12cyclictest0-21swapper/222:40:492
2105599315,10cyclictest0-21swapper/222:59:292
2105599315,10cyclictest0-21swapper/222:59:292
21055993111,17cyclictest0-21swapper/223:55:122
2104799311,19cyclictest19528-21rm23:05:131
2104799310,24cyclictest15525-21ssh00:15:121
2103699318,7cyclictest0-21swapper/020:50:260
21036993112,4cyclictest15296-21apt-get20:45:130
21036993110,7cyclictest23711-21apt-get20:20:120
2106399302,14cyclictest35966irq/58-eth021:10:123
2105599308,9cyclictest21059-21ntpq19:40:232
2105599308,8cyclictest28812-21sh00:04:062
2105599308,7cyclictest31110-21ssh21:21:572
2105599307,8cyclictest3939-21cut21:05:222
2105599307,11cyclictest9961-21ssh22:22:372
2105599306,11cyclictest0-21swapper/223:00:062
21055993020,7cyclictest30086-21apt-get00:05:172
2104799303,21cyclictest28643-21ssh21:40:141
2104799302,17cyclictest10684-21ssh21:30:121
2103699309,8cyclictest28587-21grep19:50:200
2105599299,7cyclictest6666-21diskmemload21:25:372
2105599299,17cyclictest14204-21fschecks_count20:10:162
2105599294,11cyclictest0-21swapper/220:40:022
2104799296,14cyclictest21-21ksoftirqd/121:10:291
2104799291,19cyclictest16039-21ssh21:15:151
2105599289,12cyclictest0-21swapper/220:05:112
2105599287,6cyclictest0-21swapper/222:30:452
2105599283,12cyclictest16066-21ssh00:31:552
21055992815,11cyclictest30373-21apt-get22:35:132
21055992814,12cyclictest24704-21kthreadcore20:20:202
21055992814,11cyclictest8149-21diskstats19:30:152
21055992811,13cyclictest1917-21ntp_states19:20:262
21055992810,9cyclictest27584-21irqcore19:50:182
21055992810,15cyclictest29838-21sort19:15:292
21055992711,13cyclictest4205-21cat20:30:222
21055992711,13cyclictest21165-21latency_hist19:10:022
2104799279,9cyclictest21-21ksoftirqd/122:20:441
2104799279,8cyclictest21-21ksoftirqd/123:51:171
2104799279,8cyclictest21-21ksoftirqd/123:20:551
2104799278,9cyclictest21-21ksoftirqd/123:42:161
2106399262,18cyclictest0-21swapper/323:33:403
2106399262,18cyclictest0-21swapper/323:33:403
2105599262,16cyclictest27346irq/40-dwc2_hso20:40:122
21055992610,12cyclictest29708-21/usr/sbin/munin21:00:162
21055992610,12cyclictest25951-21latency_hist19:15:012
2104799269,9cyclictest21-21ksoftirqd/123:15:131
2104799269,8cyclictest21-21ksoftirqd/121:39:311
2104799269,8cyclictest21-21ksoftirqd/100:13:031
2104799269,7cyclictest21-21ksoftirqd/121:51:431
2104799268,9cyclictest21-21ksoftirqd/100:36:421
2104799267,8cyclictest21-21ksoftirqd/123:13:441
2106399252,14cyclictest0-21swapper/323:35:143
2106399251,14cyclictest35966irq/58-eth022:50:123
21055992512,10cyclictest3886-21irqstats19:25:162
21055992510,7cyclictest25945-21grep19:45:272
21055992510,11cyclictest0-21swapper/219:35:122
2104799259,8cyclictest21-21ksoftirqd/122:53:411
2104799259,7cyclictest21-21ksoftirqd/123:36:221
2104799258,9cyclictest21-21ksoftirqd/122:06:181
2104799258,9cyclictest21-21ksoftirqd/122:06:171
2104799258,8cyclictest21-21ksoftirqd/121:26:381
2104799252,18cyclictest30118-21ssh00:05:171
21047992512,4cyclictest21-21ksoftirqd/122:02:171
2104799251,20cyclictest21010-21ssh00:00:141
2104799251,18cyclictest31754-21ssh23:30:111
2104799251,18cyclictest31754-21ssh23:30:101
21047992510,9cyclictest21-21ksoftirqd/121:59:151
21047992510,7cyclictest21-21ksoftirqd/121:49:531
21047992510,4cyclictest21-21ksoftirqd/122:57:391
21047992510,4cyclictest21-21ksoftirqd/122:57:391
21036992515,7cyclictest21166-21latency_hist19:10:020
21063992412,8cyclictest35966irq/58-eth023:45:143
21055992411,11cyclictest930-21snmpd20:04:572
2104799249,8cyclictest21-21ksoftirqd/100:25:391
2104799248,9cyclictest21-21ksoftirqd/122:27:471
2104799248,8cyclictest21-21ksoftirqd/122:10:351
2104799248,8cyclictest21-21ksoftirqd/121:22:121
2104799246,8cyclictest21-21ksoftirqd/122:31:231
2104799245,15cyclictest21-21ksoftirqd/122:45:141
2106399232,14cyclictest0-21swapper/323:25:143
2104799239,4cyclictest21-21ksoftirqd/119:45:231
2104799231,17cyclictest20345-21ls20:50:171
21047992310,4cyclictest21-21ksoftirqd/122:15:081
2106399229,8cyclictest35966irq/58-eth023:55:133
2106399229,8cyclictest35966irq/58-eth021:45:113
2106399223,12cyclictest35966irq/58-eth000:20:123
2106399222,13cyclictest0-21swapper/322:05:133
2106399222,13cyclictest0-21swapper/322:05:133
21055992211,8cyclictest21483-21tr20:50:212
2104799226,7cyclictest21-21ksoftirqd/119:55:131
2104799223,7cyclictest21-21ksoftirqd/120:20:301
21047992217,3cyclictest0-21swapper/120:35:131
21047992212,7cyclictest17386-21cron19:40:011
2104799221,18cyclictest18878-21find20:15:151
2106399215,9cyclictest35966irq/58-eth000:25:143
2106399213,12cyclictest35966irq/58-eth000:05:173
2106399213,12cyclictest0-21swapper/300:36:413
2106399211,15cyclictest0-21swapper/322:55:173
2106399211,15cyclictest0-21swapper/322:55:173
2104799219,4cyclictest21-21ksoftirqd/119:20:221
2104799215,3cyclictest131rcu_preempt21:05:241
2104799214,4cyclictest11003-21ps20:40:161
2104799213,8cyclictest131rcu_preempt21:00:131
2104799213,3cyclictest0-21swapper/120:30:011
2104799211,5cyclictest21-21ksoftirqd/120:55:281
2104799210,6cyclictest9545-21irqcore20:05:171
2106399209,8cyclictest0-21swapper/322:45:263
2106399203,12cyclictest0-21swapper/323:50:413
21063992012,5cyclictest35966irq/58-eth021:55:123
2106399201,14cyclictest0-21swapper/300:15:023
2104799206,7cyclictest21-21ksoftirqd/120:00:111
2104799203,9cyclictest21-21ksoftirqd/119:30:141
2104799203,4cyclictest21-21ksoftirqd/119:15:191
2104799201,16cyclictest6508-21ls19:25:201
2106399199,8cyclictest0-21swapper/321:26:143
2106399192,9cyclictest0-21swapper/300:30:153
2106399192,11cyclictest0-21swapper/319:20:143
21063991911,5cyclictest35966irq/58-eth023:00:133
2106399191,14cyclictest35966irq/58-eth021:35:133
2106399191,12cyclictest0-21swapper/319:55:273
2106399191,12cyclictest0-21swapper/319:50:013
2104799194,2cyclictest131rcu_preempt20:45:131
2104799193,9cyclictest21-21ksoftirqd/119:45:011
2104799192,3cyclictest0-21swapper/119:10:111
2106399189,7cyclictest0-21swapper/322:35:153
2106399189,6cyclictest35966irq/58-eth023:10:153
2106399189,6cyclictest0-21swapper/322:40:253
2106399188,8cyclictest0-21swapper/300:17:383
2106399183,12cyclictest0-21swapper/323:20:123
2106399183,10cyclictest0-21swapper/323:40:393
2106399182,10cyclictest0-21swapper/321:00:133
2106399182,10cyclictest0-21swapper/320:40:133
2106399182,10cyclictest0-21swapper/320:20:143
2106399182,10cyclictest0-21swapper/320:00:133
2106399182,10cyclictest0-21swapper/319:40:123
2106399182,10cyclictest0-21swapper/319:35:133
2106399182,10cyclictest0-21swapper/319:30:143
2106399182,10cyclictest0-21swapper/319:25:123
2106399182,10cyclictest0-21swapper/319:15:143
21063991812,4cyclictest0-21swapper/323:05:213
21063991811,4cyclictest0-21swapper/322:00:113
2106399181,13cyclictest0-21swapper/322:15:213
21063991810,6cyclictest0-21swapper/321:18:243
2104799183,3cyclictest131rcu_preempt20:30:211
21047991812,4cyclictest0-21swapper/119:50:181
2106399179,6cyclictest0-21swapper/323:15:533
2106399179,6cyclictest0-21swapper/322:20:323
2106399172,10cyclictest0-21swapper/320:45:143
2106399172,10cyclictest0-21swapper/320:30:133
2106399171,12cyclictest0-21swapper/320:05:133
2106399161,8cyclictest0-21swapper/320:35:113
2106399161,12cyclictest0-21swapper/320:55:173
2106399161,12cyclictest0-21swapper/319:10:153
2106399151,10cyclictest0-21swapper/321:05:153
2104799151,11cyclictest21079-21cat19:10:021
2106399149,3cyclictest0-21swapper/320:15:223
2106399141,11cyclictest0-21swapper/320:55:013
2106399141,11cyclictest0-21swapper/319:50:283
2106399131,7cyclictest0-21swapper/320:25:273
2106399101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional