You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 00:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Sep 19, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1494899601,49cyclictest0-21swapper/211:40:132
1494899542,39cyclictest16506-21apt-get10:30:142
1494899531,38cyclictest19875-21apt-get07:15:142
1496399504,30cyclictest35966irq/58-eth009:20:143
1494899502,30cyclictest15471-21ssh11:58:242
1496399493,28cyclictest35966irq/58-eth010:15:153
1494899491,25cyclictest23244-21ssh09:40:142
1493999480,37cyclictest636-21diskmemload12:15:131
1494899471,40cyclictest7922-21apt-get08:10:142
1496399464,27cyclictest35966irq/58-eth012:20:133
1494899461,41cyclictest23709-21apt-get09:00:122
1496399452,27cyclictest35966irq/58-eth011:50:133
1496399452,27cyclictest35966irq/58-eth008:00:123
1494899453,33cyclictest0-21swapper/209:35:132
1494899452,40cyclictest25606-21ssh10:35:152
1494899451,40cyclictest0-21swapper/208:45:142
1494899450,38cyclictest6022-21apt-get11:00:142
1494899442,33cyclictest18850-21ssh11:25:152
1494899441,31cyclictest0-21swapper/212:00:142
1494899440,39cyclictest18909-21apt-get08:55:152
1496399434,23cyclictest35966irq/58-eth011:40:133
14928994327,12cyclictest630-21Async09:36:310
1496399423,24cyclictest35966irq/58-eth012:05:123
1496399423,24cyclictest35966irq/58-eth012:05:123
1494899421,37cyclictest18095-21sh09:55:162
1494899421,36cyclictest0-21swapper/209:10:132
1496399412,25cyclictest35966irq/58-eth010:10:133
1494899411,34cyclictest0-21swapper/210:55:142
1494899411,27cyclictest30218-21ssh12:25:142
1494899410,34cyclictest0-21swapper/207:55:132
1496399405,23cyclictest35966irq/58-eth009:40:123
1496399403,22cyclictest35966irq/58-eth009:35:153
1494899401,33cyclictest2999-21apt-get12:10:142
1493999402,30cyclictest5121-21ssh09:30:141
1494899391,32cyclictest28519-21apt-get09:05:142
1494899390,34cyclictest0-21swapper/212:05:122
1494899390,34cyclictest0-21swapper/212:05:122
1496399382,21cyclictest35966irq/58-eth012:35:143
1494899381,33cyclictest16026-21ssh12:35:142
1494899381,22cyclictest0-21swapper/207:40:162
1494899380,32cyclictest21159-21ssh12:20:122
1494899371,33cyclictest0-21swapper/211:05:132
1494899371,30cyclictest8942-21apt-get09:50:132
1492899374,25cyclictest6051-21ssh11:00:140
14963993621,8cyclictest35966irq/58-eth009:45:123
1494899365,22cyclictest22455-21ssh11:45:142
1494899361,31cyclictest0-21swapper/208:35:122
1496399355,21cyclictest35966irq/58-eth010:30:143
1496399354,19cyclictest35966irq/58-eth011:30:163
1494899355,23cyclictest24123-21ssh11:10:152
1494899352,24cyclictest0-21swapper/209:45:132
1494899351,30cyclictest0-21swapper/211:50:132
1494899351,30cyclictest0-21swapper/208:05:142
1494899351,28cyclictest27346irq/40-dwc2_hso10:15:142
1494899350,31cyclictest3709-21apt-get10:05:132
1494899350,30cyclictest0-21swapper/210:10:152
1494899350,29cyclictest0-21swapper/209:25:122
1494899350,26cyclictest19437-21cut07:45:232
1494899350,22cyclictest9504-21sh10:43:252
1493999350,29cyclictest21785-21ssh10:15:141
1494899342,28cyclictest29515-21apt-get07:25:132
1494899341,31cyclictest18685-21munin-run08:55:012
1494899341,30cyclictest7829-21ssh11:18:092
1494899341,30cyclictest5372-21ntp_states07:30:252
1494899341,29cyclictest27296-21df_inode08:30:152
1494899340,31cyclictest0-21swapper/210:00:152
1494899340,30cyclictest16658-21ssh09:17:202
1494899340,27cyclictest3969-21ssh11:34:572
1494899340,27cyclictest27346irq/40-dwc2_hso08:40:162
1494899340,26cyclictest9736-21ssh11:36:042
1494899340,26cyclictest9736-21ssh11:36:042
1494899340,25cyclictest16393-21ssh11:22:352
1494899340,24cyclictest11777-21ssh09:32:382
1494899340,22cyclictest0-21swapper/207:20:132
1493999341,29cyclictest9820-21ssh11:20:151
14963993323,8cyclictest0-21swapper/311:15:133
1494899332,14cyclictest17685-21ssh12:16:242
1494899331,27cyclictest0-21swapper/208:00:122
1494899330,26cyclictest0-21swapper/210:50:132
1493999331,24cyclictest27862-21ssh11:30:161
14963993216,12cyclictest35966irq/58-eth011:10:153
14963993215,8cyclictest35966irq/58-eth010:35:143
1494899321,27cyclictest15089-21apt-get07:10:132
1494899320,29cyclictest21131-21acpi07:50:132
1494899320,29cyclictest11238-21/usr/sbin/munin07:35:302
1494899320,26cyclictest7417-21apt-get10:25:142
1494899312,21cyclictest26217-21unixbench_multi08:25:282
1494899310,28cyclictest30648-21ssh10:20:022
1494899310,28cyclictest30648-21ssh10:20:022
14963993014,13cyclictest0-21swapper/309:30:143
1494899300,27cyclictest13358-21ssh12:32:202
1494899300,26cyclictest24687-21ssh09:21:032
1494899300,26cyclictest17242-21awk08:15:312
1494899300,23cyclictest17963-21ssh10:47:422
1492899302,22cyclictest20235-21ssh12:19:240
1492899302,19cyclictest8974-21ssh09:50:130
1494899291,24cyclictest0-21swapper/208:20:142
1493999294,15cyclictest21-21ksoftirqd/108:55:141
1492899292,21cyclictest5926-21ssh12:29:240
1492899292,19cyclictest7447-21ssh10:25:140
14928992920,6cyclictest12-21ksoftirqd/011:45:140
1496399283,12cyclictest0-21swapper/312:00:243
1496399273,12cyclictest0-21swapper/312:15:133
1496399271,12cyclictest35966irq/58-eth010:40:153
1496399265,11cyclictest0-21swapper/312:10:153
1493999262,4cyclictest2235-21apt-get10:40:151
1493999262,18cyclictest29433-21ssh10:55:161
14928992615,7cyclictest12-21ksoftirqd/009:15:150
1496399253,15cyclictest0-21swapper/311:20:153
14963992510,9cyclictest35966irq/58-eth010:20:143
14963992510,9cyclictest35966irq/58-eth010:20:143
1493999256,12cyclictest21-21ksoftirqd/111:55:161
1493999253,15cyclictest4767-21kworker/u8:209:55:161
1493999252,18cyclictest20505-21ssh12:00:241
1496399243,15cyclictest35966irq/58-eth011:05:123
1493999241,20cyclictest21-21ksoftirqd/109:50:121
1492899243,7cyclictest131rcu_preempt07:40:120
1496399234,10cyclictest35966irq/58-eth009:50:113
1496399233,16cyclictest0-21swapper/312:29:243
1493999231,17cyclictest32303-21apt-get09:45:121
1493999231,17cyclictest20304-21ssh10:50:151
1493999231,17cyclictest19367-21ssh09:20:151
1493999231,16cyclictest4464-21ssh11:35:151
1493999231,16cyclictest4464-21ssh11:35:151
1493999231,16cyclictest27174-21ssh10:00:161
14928992314,6cyclictest14199-21ssh09:35:140
14928992313,7cyclictest12-21ksoftirqd/009:05:130
1496399223,11cyclictest0-21swapper/309:00:123
1496399221,14cyclictest35966irq/58-eth011:35:163
1496399221,14cyclictest35966irq/58-eth011:35:153
1493999222,14cyclictest24759-21ntp_192.168.11412:20:241
1493999221,17cyclictest30839-21ssh10:20:141
1493999221,17cyclictest30839-21ssh10:20:141
1493999221,15cyclictest29699-21uniq07:25:171
1493999221,15cyclictest15111-21ssh11:05:131
1493999221,15cyclictest10275-21ssh09:15:161
1493999221,10cyclictest23142-21irqstats11:45:191
1492899225,10cyclictest1133-21apt-get09:10:120
14928992212,7cyclictest1-21systemd10:30:250
1492899221,17cyclictest32321-21ssh09:45:130
1496399217,10cyclictest0-21swapper/312:30:163
1496399215,11cyclictest35966irq/58-eth009:15:163
1496399212,12cyclictest0-21swapper/310:50:143
14963992113,5cyclictest0-21swapper/310:55:233
14963992112,6cyclictest0-21swapper/309:10:143
14939992112,3cyclictest636-21diskmemload12:30:011
1493999211,14cyclictest2167-21ls07:30:161
1493999211,13cyclictest27082-21ls08:30:011
1493999211,12cyclictest18715-21latency_hist08:55:011
14939992110,7cyclictest22479-21ntpq11:25:271
1492899211,17cyclictest19030-21ls11:05:290
1492899210,13cyclictest20008-21uniq07:15:160
1496399209,8cyclictest0-21swapper/311:59:063
1496399202,15cyclictest0-21swapper/309:25:133
1496399202,13cyclictest33-21ksoftirqd/310:45:133
1496399202,11cyclictest35966irq/58-eth011:25:163
14963992010,7cyclictest0-21swapper/310:00:303
1493999205,6cyclictest11776-21cat12:30:321
1493999201,9cyclictest11266-21cron07:40:011
1492899203,14cyclictest27176-21cpu10:00:160
1492899203,13cyclictest3024-21ssh12:10:150
1492899201,4cyclictest0-21swapper/009:57:310
14928992014,4cyclictest12-21ksoftirqd/012:00:150
14928992013,5cyclictest0-21swapper/010:10:280
1492899201,11cyclictest2948-21kthreadcore11:15:220
1496399199,8cyclictest0-21swapper/310:25:083
14963991912,4cyclictest0-21swapper/310:05:153
1496399191,13cyclictest0-21swapper/308:10:173
1496399191,11cyclictest0-21swapper/308:55:183
14963991910,6cyclictest0-21swapper/309:59:543
1494899190,17cyclictest27346irq/40-dwc2_hso07:10:022
1493999199,8cyclictest26456-21df_inode12:05:141
1493999199,8cyclictest26456-21df_inode12:05:141
1493999193,5cyclictest21-21ksoftirqd/110:40:011
1493999192,4cyclictest21-21ksoftirqd/110:05:291
1493999191,5cyclictest8328-21kthreadcore10:25:191
14939991913,4cyclictest0-21swapper/108:45:291
1493999191,12cyclictest16539-21ssh10:30:141
14939991910,7cyclictest7783-21memory08:40:251
14939991910,7cyclictest13584-21forks11:40:161
14939991910,7cyclictest0-21swapper/107:55:291
1493999190,13cyclictest32133-21uniq08:35:161
1492899199,2cyclictest91-21pr/ttyS010:45:010
1492899198,8cyclictest31996-21apt-get08:35:130
1492899198,8cyclictest29255-21ntpq10:35:270
1492899192,13cyclictest24133-21ssh11:10:150
1492899192,11cyclictest24764-21ssh12:20:240
1492899191,4cyclictest0-21swapper/012:05:020
1492899191,4cyclictest0-21swapper/012:05:020
14928991913,3cyclictest0-21swapper/011:50:150
1492899191,16cyclictest28458-21ssh09:25:130
1492899191,13cyclictest5149-21sh09:30:160
1492899191,10cyclictest20805-21irqcore10:50:190
14928991910,6cyclictest12728-21apt-get08:15:140
1496399189,6cyclictest0-21swapper/311:00:503
1496399182,10cyclictest0-21swapper/308:35:143
1496399182,10cyclictest0-21swapper/308:25:043
1496399182,10cyclictest0-21swapper/308:20:163
1496399182,10cyclictest0-21swapper/308:15:143
1496399182,10cyclictest0-21swapper/307:20:143
1496399182,10cyclictest0-21swapper/307:15:143
1496399181,13cyclictest0-21swapper/308:50:273
1493999189,7cyclictest1-21systemd12:10:461
1493999188,7cyclictest9820-21cron09:15:011
1493999182,12cyclictest4503-21ntp_states11:15:271
1493999181,8cyclictest28424-21/usr/sbin/munin09:25:161
1493999181,6cyclictest16141-21df12:35:161
14939991811,5cyclictest8029-21kthreadcore11:00:211
14939991811,5cyclictest0-21swapper/108:20:301
1493999181,12cyclictest14985-21ntp_states10:45:261
14939991810,6cyclictest167-21systemd-network11:51:161
1492899189,7cyclictest8350-21cat10:05:300
1492899189,7cyclictest28259-21users09:00:300
1492899189,7cyclictest12385-21ssh10:45:200
1492899182,14cyclictest26831-21kthreadcore07:55:200
1492899182,13cyclictest31527-21awk11:30:290
1492899182,13cyclictest21800-21ssh10:15:150
1492899182,12cyclictest17178-21ssh11:40:270
1492899182,11cyclictest0-21swapper/007:25:150
1492899181,4cyclictest0-21swapper/009:20:590
14928991812,4cyclictest0-21swapper/011:40:010
14928991812,4cyclictest0-21swapper/011:40:010
1492899181,14cyclictest4131-21apt-get08:40:040
14928991811,4cyclictest0-21swapper/009:40:120
1492899181,13cyclictest21650-21kthreadcore11:25:230
1492899181,13cyclictest21148-21apt-get07:50:140
1492899181,11cyclictest19646-21ssh12:35:260
1496399179,6cyclictest0-21swapper/311:45:103
1496399172,10cyclictest0-21swapper/308:40:053
1496399171,13cyclictest0-21swapper/307:40:143
1493999179,6cyclictest167-21systemd-network08:02:141
1493999175,10cyclictest23663-21kthreadcore07:50:231
1493999171,8cyclictest19777-21date07:15:021
1493999171,12cyclictest31929-21munin-run09:45:011
1493999171,12cyclictest12737-21apt-get10:10:131
1492899179,6cyclictest14584-21cat11:20:320
1492899178,6cyclictest9269-21apt-get08:45:150
1492899178,3cyclictest31779-21cut08:00:180
1492899177,7cyclictest29406-21apt-get10:55:150
1492899172,8cyclictest0-21swapper/007:10:130
1492899171,9cyclictest7050-21diskstats12:30:150
14928991712,3cyclictest5362-21ntp_offset07:30:240
14928991711,4cyclictest0-21swapper/007:45:250
1492899171,13cyclictest30946-21diskstats10:20:150
1492899171,13cyclictest30946-21diskstats10:20:150
1496399161,12cyclictest0-21swapper/307:35:273
1493999169,5cyclictest13206-21sed08:15:181
1493999169,5cyclictest0-21swapper/108:05:261
1493999162,10cyclictest0-21swapper/108:10:151
1493999161,13cyclictest26138-21kthreadcore07:20:191
1493999161,12cyclictest14189-21apt-get09:35:131
1493999161,11cyclictest28469-21kthreadcore08:30:201
1493999161,11cyclictest27361-21kthreadcore11:10:241
1492899167,7cyclictest24813-21df07:20:140
1492899165,8cyclictest6933-21grep07:35:160
1492899165,8cyclictest12-21ksoftirqd/008:10:150
1492899164,10cyclictest21211-21kthreadcore08:55:240
1492899163,10cyclictest25973-21mailstats08:25:270
1492899163,10cyclictest20718-21kthreadcore08:20:240
1492899162,11cyclictest6585-21sshd08:05:250
1492899161,12cyclictest28324-21kthreadcore08:30:190
1496399154,5cyclictest0-21swapper/307:55:143
1493999158,5cyclictest17275-21kthreadcore07:45:201
1493999157,6cyclictest23660-21cat07:15:281
1493999152,10cyclictest25635-21kthreadcore09:00:211
1493999151,11cyclictest29998-21kthreadcore09:05:201
1493999151,10cyclictest11504-21/usr/sbin/munin07:40:321
1492899153,9cyclictest844-21munin-node08:50:130
1492899151,11cyclictest15032-21latency_hist07:10:020
1496399141,11cyclictest0-21swapper/309:05:233
1496399141,10cyclictest0-21swapper/307:10:183
1493999147,4cyclictest15037-21grep07:10:031
1496399133,7cyclictest321rcuc/307:45:143
1496399131,7cyclictest0-21swapper/307:25:283
1496399131,6cyclictest0-21swapper/308:05:133
1496399131,10cyclictest0-21swapper/308:50:023
1496399131,10cyclictest0-21swapper/307:30:263
1496399127,3cyclictest0-21swapper/308:30:033
1496399126,3cyclictest0-21swapper/307:50:153
1496399111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional