You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-23 - 18:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Jun 23, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1572399701,12cyclictest0-21swapper/207:40:122
1570799673,50cyclictest29946-21ssh10:55:140
1570799621,55cyclictest10943-21apt-get09:15:130
15707995919,32cyclictest536-21usb-storage09:05:100
15723995621,17cyclictest27-21ksoftirqd/210:55:132
1570799546,6cyclictest131rcu_preempt07:40:130
1571599538,5cyclictest131rcu_preempt10:50:131
1570799524,33cyclictest31535-21ssh11:50:110
1572399491,35cyclictest26042-21ssh12:05:152
1570799492,42cyclictest22593-21apt-get11:45:150
1572399481,37cyclictest8036-21ssh10:25:152
1573299474,27cyclictest35966irq/58-eth011:20:113
1570799461,39cyclictest10124-21apt-get08:45:130
1570799461,37cyclictest8097-21apt-get11:55:110
1570799460,42cyclictest0-21swapper/012:10:130
1570799460,42cyclictest0-21swapper/012:10:120
1571599453,6cyclictest131rcu_preempt09:55:121
1571599452,32cyclictest20535-21ssh12:20:161
1570799451,37cyclictest4564-21apt-get11:35:130
1570799442,35cyclictest13153-21apt-get10:10:120
1570799440,40cyclictest24500-21apt-get09:00:100
15723994327,10cyclictest12863-21ssh11:39:402
1571599432,29cyclictest31361-21ssh10:20:131
1571599432,21cyclictest15025-21ssh12:35:131
1570799431,38cyclictest17083-21apt-get12:00:120
1570799422,35cyclictest9397-21apt-get09:50:130
1570799422,35cyclictest9397-21apt-get09:50:120
1573299417,18cyclictest35966irq/58-eth011:25:123
1573299413,23cyclictest35966irq/58-eth010:40:123
1573299413,23cyclictest35966irq/58-eth010:40:113
1570799412,21cyclictest0-21swapper/010:44:450
1570799412,21cyclictest0-21swapper/010:44:440
1570799411,26cyclictest10704-21kthreadcore10:25:220
1570799410,28cyclictest325-21ssh11:14:390
1570799410,28cyclictest325-21ssh11:14:380
1570799410,27cyclictest1508-21diskmemload11:34:230
1573299404,21cyclictest35966irq/58-eth009:20:123
1571599401,35cyclictest6464-21ssh11:00:131
1570799403,32cyclictest340-21apt-get09:45:130
1570799401,35cyclictest31628-21apt-get08:00:130
1570799401,35cyclictest14998-21apt-get12:35:120
1570799401,34cyclictest6452-21apt-get11:00:120
1570799401,32cyclictest23218-21apt-get08:25:130
1573299395,25cyclictest35966irq/58-eth010:35:123
1572399392,19cyclictest13775-21ssh09:34:282
1571599391,34cyclictest29500-21rm12:25:131
1570799391,34cyclictest23715-21apt-get09:40:130
1570799391,33cyclictest4097-21apt-get10:05:120
1570799391,28cyclictest29009-21ssh10:17:420
1570799390,34cyclictest14907-21apt-get08:50:120
1570799390,33cyclictest8794-21apt-get08:10:130
1570799390,25cyclictest24177-21ssh09:56:330
1570799390,24cyclictest27520-21ssh12:23:070
1573299383,22cyclictest35966irq/58-eth011:30:113
1571599382,32cyclictest19076-21ssh11:25:111
1570799382,31cyclictest0-21swapper/009:10:120
1570799381,33cyclictest0-21swapper/012:30:120
1570799381,24cyclictest13581-21/usr/sbin/munin08:15:270
1570799381,22cyclictest0-21swapper/011:43:140
1570799380,32cyclictest26719-21apt-get07:55:120
1570799380,31cyclictest20022-21apt-get09:20:140
1570799380,24cyclictest8589-21sshd11:18:390
1572399372,29cyclictest22965-21ssh11:08:392
1572399372,18cyclictest8567-21ssh09:49:292
1571599376,22cyclictest1508-21diskmemload11:40:141
1570799371,32cyclictest11763-21ssh10:45:140
1570799371,31cyclictest5242-21apt-get08:40:120
1570799371,31cyclictest15384-21apt-get11:05:100
1570799371,21cyclictest0-21swapper/010:36:250
1570799370,33cyclictest0-21swapper/008:05:120
1570799370,25cyclictest19968-21ssh09:36:020
1572399363,25cyclictest0-21swapper/209:40:392
15715993618,15cyclictest21-21ksoftirqd/111:10:151
15715993618,15cyclictest21-21ksoftirqd/111:10:141
1570799360,33cyclictest20623-21apt-get07:15:130
1570799360,32cyclictest29475-21apt-get12:25:140
1570799360,32cyclictest10115-21apt-get11:20:120
1570799360,23cyclictest10734-21ssh09:30:510
15732993517,13cyclictest35966irq/58-eth012:10:133
15732993517,13cyclictest35966irq/58-eth012:10:133
15732993512,11cyclictest0-21swapper/309:50:133
15732993512,11cyclictest0-21swapper/309:50:123
1572399358,21cyclictest27-21ksoftirqd/210:02:392
1572399353,22cyclictest22269-21ssh10:15:152
1572399352,19cyclictest27346irq/40-dwc2_hso12:25:122
1571599359,5cyclictest131rcu_preempt10:30:131
1571599353,28cyclictest370-21ssh09:45:131
1571599353,21cyclictest8124-21ssh11:55:111
1573299343,20cyclictest35966irq/58-eth010:30:133
1573299342,20cyclictest35966irq/58-eth009:15:123
1573299342,19cyclictest35966irq/58-eth009:45:133
1572399341,26cyclictest15024-21ssh12:35:142
1571599342,24cyclictest20049-21ssh09:20:131
1571599342,24cyclictest10970-21ssh09:15:141
1571599341,27cyclictest11527-21ssh12:15:121
1570799341,28cyclictest28024-21apt-get08:30:100
1570799340,3cyclictest17088-21ssh10:30:140
1573299333,19cyclictest35966irq/58-eth008:05:123
1573299331,22cyclictest35966irq/58-eth012:20:153
1573299331,22cyclictest35966irq/58-eth012:00:133
1572399333,7cyclictest0-21swapper/211:33:382
1572399332,15cyclictest21465-21sh10:14:332
1570799331,29cyclictest0-21swapper/010:50:130
1570799331,28cyclictest26173-21fschecks_count12:05:170
1570799330,21cyclictest15846-21/usr/sbin/munin07:10:250
1572399321,24cyclictest0-21swapper/211:20:112
1572399321,17cyclictest0-21swapper/211:25:132
1570799321,29cyclictest19365-21idleruntime11:25:150
1570799321,28cyclictest7631-21hwlatdetect07:35:150
1570799321,27cyclictest1421-21tr07:25:270
1570799320,28cyclictest29175-21timerandwakeup07:20:270
1570799320,28cyclictest29074-21ssh09:25:120
1570799320,28cyclictest17254-21fw_packets07:45:160
1573299314,14cyclictest35966irq/58-eth010:45:143
1573299312,18cyclictest35966irq/58-eth011:35:123
1572399315,18cyclictest27346irq/40-dwc2_hso12:00:122
1572399315,12cyclictest1081-21ssh11:15:122
1572399311,14cyclictest15551-21ssh11:58:392
1572399311,11cyclictest0-21swapper/208:00:132
15715993112,10cyclictest21-21ksoftirqd/108:25:131
1570799310,28cyclictest5000-21ssh10:21:550
1570799310,28cyclictest4966-21sshd08:35:400
1570799310,28cyclictest4966-21sshd08:35:390
1570799310,28cyclictest31814-21cat10:00:290
1570799310,27cyclictest2813-21tune2fs07:30:170
1570799310,27cyclictest22278-21head08:20:280
1573299305,16cyclictest35966irq/58-eth011:55:113
1572399303,15cyclictest27346irq/40-dwc2_hso09:35:122
1571599303,20cyclictest11782-21ssh10:45:141
1571599301,23cyclictest17100-21ssh12:00:141
1570799301,25cyclictest15679-21sed12:15:300
1570799301,25cyclictest0-21swapper/008:55:100
1571599292,22cyclictest23746-21ssh09:40:131
1570799291,25cyclictest25294-21ntpq07:50:260
1570799290,26cyclictest15743-21latency_hist07:10:020
1573299283,18cyclictest35966irq/58-eth012:35:133
1573299283,18cyclictest0-21swapper/310:15:383
1572399287,5cyclictest27-21ksoftirqd/212:20:152
1572399286,7cyclictest27-21ksoftirqd/210:21:392
1572399283,19cyclictest0-21swapper/210:42:392
1572399283,19cyclictest0-21swapper/210:42:392
1572399282,20cyclictest511-21ssh11:14:502
1572399282,20cyclictest511-21ssh11:14:502
15723992817,6cyclictest27-21ksoftirqd/207:45:122
15723992813,11cyclictest22618-21sh11:45:162
1571599285,17cyclictest630-21Async09:36:311
1571599285,14cyclictest27438-21ssh10:00:131
1571599282,20cyclictest21466-21rm10:14:341
1573299274,15cyclictest35966irq/58-eth011:45:143
1573299273,17cyclictest0-21swapper/309:34:273
1572399279,13cyclictest0-21swapper/208:25:132
1572399275,11cyclictest27346irq/40-dwc2_hso12:30:122
1572399273,18cyclictest0-21swapper/210:49:442
1572399272,20cyclictest0-21swapper/209:05:112
1572399272,12cyclictest27-21ksoftirqd/208:20:132
15723992711,8cyclictest27-21ksoftirqd/207:20:272
15723992711,5cyclictest0-21swapper/210:31:412
1572399271,11cyclictest26105-21ssh10:35:132
15723992710,13cyclictest2730-21grep12:10:162
15723992710,13cyclictest2730-21grep12:10:162
1571599275,10cyclictest131rcu_preempt09:50:121
1571599275,10cyclictest131rcu_preempt09:50:121
1571599272,19cyclictest31904-21ssh10:36:381
1571599271,20cyclictest0-21swapper/107:25:131
1573299262,16cyclictest0-21swapper/311:08:393
1572399269,5cyclictest27-21ksoftirqd/209:58:442
1572399268,7cyclictest27-21ksoftirqd/211:01:442
1572399267,11cyclictest27-21ksoftirqd/211:51:462
1572399264,16cyclictest27346irq/40-dwc2_hso07:55:132
1572399264,13cyclictest27346irq/40-dwc2_hso07:50:132
1572399262,9cyclictest0-21swapper/207:10:142
1571599262,19cyclictest15410-21ssh11:05:111
15715992614,6cyclictest0-21swapper/111:15:121
1571599261,18cyclictest2698-21ssh10:40:111
1571599261,18cyclictest2698-21ssh10:40:111
1573299253,15cyclictest0-21swapper/310:14:333
1573299252,17cyclictest0-21swapper/309:40:123
1573299252,14cyclictest35966irq/58-eth011:50:113
1572399254,14cyclictest27-21ksoftirqd/209:50:502
1572399254,14cyclictest27-21ksoftirqd/209:50:502
1572399251,20cyclictest19639-21latency_hist09:20:012
15723992511,8cyclictest27-21ksoftirqd/209:28:422
1571599254,13cyclictest0-21swapper/108:45:151
1571599252,18cyclictest13948-21ssh09:34:391
1571599251,19cyclictest22620-21ssh11:45:151
1572399248,3cyclictest27-21ksoftirqd/209:23:012
1572399243,12cyclictest27-21ksoftirqd/210:05:232
1572399243,10cyclictest930-21snmpd07:35:112
1572399242,7cyclictest2540-21kworker/u8:009:10:052
15723992410,4cyclictest27-21ksoftirqd/208:15:012
1573299233,9cyclictest0-21swapper/311:00:143
15732992310,9cyclictest35966irq/58-eth009:10:133
15723992312,4cyclictest17553-21threads11:40:282
1572399231,17cyclictest24808-21switchtime10:50:292
1571599231,8cyclictest7634-21ssh11:54:551
15715992314,3cyclictest0-21swapper/108:00:121
1573299223,12cyclictest0-21swapper/308:45:133
1571599221,9cyclictest4250-21unixbench_multi08:35:301
1571599221,9cyclictest4250-21unixbench_multi08:35:291
1571599221,10cyclictest13311-21kthreadcore11:20:211
1573299213,12cyclictest35966irq/58-eth007:25:123
1573299212,13cyclictest35966irq/58-eth012:05:143
1573299212,12cyclictest35966irq/58-eth011:40:123
1573299212,12cyclictest0-21swapper/309:35:123
15732992113,6cyclictest0-21swapper/310:51:293
1572399217,7cyclictest27-21ksoftirqd/208:15:172
1572399217,3cyclictest131rcu_preempt09:00:282
1572399211,3cyclictest0-21swapper/208:49:212
1572399211,15cyclictest12717-21kthreadcore12:15:212
1572399211,10cyclictest1666-21grep07:25:282
1571599212,14cyclictest11617-21ssh10:08:391
1571599211,9cyclictest5060-21irqcore11:35:171
1571599211,16cyclictest32665-21ls08:35:021
1571599211,15cyclictest26547-21sed07:55:011
15715992110,8cyclictest4062-21sshd11:34:531
1573299209,6cyclictest0-21swapper/312:28:083
1573299203,10cyclictest0-21swapper/309:58:383
15732992010,8cyclictest0-21swapper/311:11:533
15732992010,8cyclictest0-21swapper/311:11:523
15732992010,7cyclictest35966irq/58-eth010:25:143
15732992010,5cyclictest0-21swapper/312:30:133
1572399207,5cyclictest22755-21kthreadcore07:15:202
1572399203,9cyclictest5630-21kthreadcore08:05:202
1572399202,9cyclictest8879-21switchtime08:40:282
1572399201,9cyclictest31795-21timerwakeupswit08:30:272
1572399201,17cyclictest0-21swapper/207:30:292
1572399201,10cyclictest20035-21tr08:55:162
1571599203,14cyclictest23033-21missed_timers08:55:221
1571599201,5cyclictest14635-21munin-run12:35:021
1571599201,13cyclictest29725-21ssh12:05:271
1573299193,11cyclictest0-21swapper/311:18:393
1573299193,11cyclictest0-21swapper/310:20:403
15732991912,5cyclictest0-21swapper/310:55:273
15732991910,6cyclictest35966irq/58-eth009:25:143
1572399191,14cyclictest5150-21cat08:40:022
1572399191,14cyclictest5150-21cat08:40:012
1572399190,10cyclictest14910-21cpu08:50:142
15715991911,6cyclictest6020-21awk07:30:231
1571599191,15cyclictest16690-21latency_hist10:30:011
1571599191,14cyclictest17204-21cut07:45:171
1573299189,6cyclictest0-21swapper/312:16:013
1573299183,10cyclictest0-21swapper/307:30:153
1573299182,11cyclictest0-21swapper/310:09:323
1573299182,11cyclictest0-21swapper/310:02:393
1573299181,13cyclictest0-21swapper/308:35:283
1573299181,13cyclictest0-21swapper/308:35:283
1573299181,13cyclictest0-21swapper/307:55:273
1573299181,12cyclictest0-21swapper/308:20:273
1572399181,8cyclictest15772-21cat07:10:022
1571599189,6cyclictest6394-21cat09:10:301
1571599185,4cyclictest131rcu_preempt09:10:021
1571599183,13cyclictest1436-21ssh10:55:281
1571599183,12cyclictest2954-21ps12:10:191
1571599183,12cyclictest2954-21ps12:10:181
1571599182,4cyclictest53646usb-storage07:55:051
1571599181,15cyclictest30205-21kthreadcore09:25:211
1571599181,14cyclictest9068-21kthreadcore07:35:201
1571599181,14cyclictest23428-21tr10:15:201
1571599181,14cyclictest21951-21kthreadcore07:15:201
1573299172,9cyclictest0-21swapper/308:30:123
1573299172,9cyclictest0-21swapper/308:00:053
1573299172,9cyclictest0-21swapper/307:50:143
1573299172,9cyclictest0-21swapper/307:35:133
1573299172,10cyclictest0-21swapper/308:50:123
1573299172,10cyclictest0-21swapper/308:40:133
1571599179,5cyclictest0-21swapper/108:10:181
1571599172,13cyclictest16570-21kthreadcore07:10:191
1571599172,10cyclictest0-21swapper/109:00:121
15715991711,4cyclictest14238-21kernelversion08:15:191
1571599171,13cyclictest27547-21cut07:20:191
1573299162,9cyclictest0-21swapper/308:15:143
1573299162,9cyclictest0-21swapper/307:10:133
1571599165,8cyclictest614-21dbus-daemon08:24:131
1571599162,10cyclictest0-21swapper/108:50:121
1571599161,13cyclictest4662-21kthreadcore08:05:201
1571599161,11cyclictest8083-21kthreadcore08:40:211
1573299152,9cyclictest0-21swapper/309:05:053
1573299152,9cyclictest0-21swapper/308:10:143
1573299151,8cyclictest0-21swapper/308:25:173
1573299151,8cyclictest0-21swapper/307:15:273
1573299151,7cyclictest0-21swapper/309:00:103
1571599151,11cyclictest15768-21latency_hist07:10:031
1571599151,11cyclictest12253-21apt-get07:40:121
1573299141,7cyclictest0-21swapper/308:55:133
1573299141,10cyclictest0-21swapper/307:20:183
1573299130,9cyclictest0-21swapper/307:45:133
1573299121,5cyclictest0-21swapper/307:40:203
1573299101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional