You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 02:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Apr 26, 2026 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25135996647,13cyclictest16983-21sh23:00:152
25147996522,29cyclictest35966irq/58-eth000:00:133
25115995912,13cyclictest131rcu_preempt19:25:130
2511599563,47cyclictest17000-21ssh23:00:150
25135995416,19cyclictest27-21ksoftirqd/223:55:122
2511599543,34cyclictest30198-21ssh23:25:120
25115995345,6cyclictest0-21swapper/021:30:140
25115995234,8cyclictest14785-21ssh00:10:130
25115995234,8cyclictest14785-21ssh00:10:130
25135995026,14cyclictest27-21ksoftirqd/222:35:132
2512499481,36cyclictest5124-21ssh22:00:151
2512499471,36cyclictest10180-21ssh21:45:151
2511599465,12cyclictest131rcu_preempt19:40:130
25135994525,13cyclictest0-21swapper/221:20:142
2511599454,7cyclictest131rcu_preempt20:55:120
2514799445,25cyclictest35966irq/58-eth021:45:133
2513599445,27cyclictest27-21ksoftirqd/223:20:132
2511599441,33cyclictest18885-21ssh00:30:140
2514799412,22cyclictest35966irq/58-eth023:50:133
2514799412,22cyclictest35966irq/58-eth023:50:133
2512499412,32cyclictest1776-21ssh23:45:141
2512499412,32cyclictest1776-21ssh23:45:141
2511599401,28cyclictest10700-21diskmemload21:40:130
2513599395,23cyclictest261rcuc/200:05:142
2512499391,33cyclictest7845-21ssh22:55:131
2513599381,29cyclictest18894-21ssh00:30:152
2513599381,24cyclictest131rcu_preempt00:37:442
25124993825,11cyclictest22104-21apt-get22:45:131
2511599384,30cyclictest15885-21ssh23:35:120
2511599382,28cyclictest21047-21ssh23:20:130
2513599372,18cyclictest20705-21sh00:11:442
2513599372,18cyclictest20705-21sh00:11:442
2511599371,22cyclictest25054-21ssh23:40:140
25124993616,15cyclictest0-21swapper/120:35:111
2514799355,19cyclictest35966irq/58-eth021:20:123
2512499353,26cyclictest19265-21sh21:50:141
25124993523,10cyclictest19311-21apt-get20:45:141
2514799349,22cyclictest35966irq/58-eth021:35:133
2514799346,20cyclictest35966irq/58-eth021:05:133
2514799345,2cyclictest0-21swapper/321:55:123
2514799344,17cyclictest35966irq/58-eth022:25:123
25124993419,10cyclictest24339-21ssh21:35:121
2512499341,25cyclictest15290-21sh21:30:141
2511599342,24cyclictest10170-21ssh21:45:140
2511599341,25cyclictest31195-21ssh22:50:140
2514799331,6cyclictest0-21swapper/323:40:143
25135993318,10cyclictest27-21ksoftirqd/223:40:142
2513599331,27cyclictest14284-21ssh22:05:142
25124993315,12cyclictest0-21swapper/122:10:131
2511599331,27cyclictest662-21ssh00:20:150
2514799322,18cyclictest35966irq/58-eth022:10:133
25124993223,7cyclictest16973-21apt-get23:00:161
25124993223,3cyclictest2683-21apt-get23:10:131
25124993220,9cyclictest634-21apt-get00:20:141
2513599314,15cyclictest27346irq/40-dwc2_hso23:05:142
2513599313,18cyclictest11810-21kworker/2:123:15:122
2514799303,2cyclictest0-21swapper/321:50:143
2514799302,5cyclictest0-21swapper/322:55:143
2514799302,17cyclictest35966irq/58-eth021:10:143
25124993023,3cyclictest11885-21apt-get23:15:131
25124993018,9cyclictest6877-21ssh23:30:141
25124993017,9cyclictest13051-21apt-get22:40:131
2513599297,16cyclictest12992-21kworker/2:120:05:142
2513599295,13cyclictest27346irq/40-dwc2_hso19:30:132
2513599293,14cyclictest27346irq/40-dwc2_hso19:10:132
2512499299,14cyclictest25033-21apt-get23:40:131
2512499294,9cyclictest16025-21cat21:10:331
25124992923,2cyclictest18041-21apt-get20:10:131
25124992920,4cyclictest2429-21apt-get19:20:121
2511599291,7cyclictest536-21usb-storage20:25:040
2514799289,15cyclictest35966irq/58-eth020:15:133
2514799281,4cyclictest0-21swapper/319:30:143
25147992810,11cyclictest0-21swapper/322:15:123
2513599284,16cyclictest27346irq/40-dwc2_hso00:15:122
2513599283,17cyclictest27346irq/40-dwc2_hso19:35:142
2513599283,17cyclictest27346irq/40-dwc2_hso19:35:142
2513599282,21cyclictest19288-21ssh21:50:152
25135992817,8cyclictest27-21ksoftirqd/200:20:132
2513599281,17cyclictest13452-21ssh00:25:252
2512499284,10cyclictest31238-21/usr/sbin/munin19:50:181
25124992818,6cyclictest26085-21apt-get23:05:131
25124992815,9cyclictest14774-21apt-get00:10:141
25124992815,9cyclictest14774-21apt-get00:10:141
2512499281,23cyclictest23938-21ssh00:15:141
2511599282,8cyclictest131rcu_preempt00:37:430
2511599281,22cyclictest20306-21ssh21:15:150
2514799277,4cyclictest0-21swapper/321:44:093
2514799276,6cyclictest0-21swapper/323:15:263
2514799273,15cyclictest0-21swapper/300:11:443
2514799273,15cyclictest0-21swapper/300:11:443
2513599274,13cyclictest27346irq/40-dwc2_hso20:25:132
2513599273,19cyclictest27346irq/40-dwc2_hso21:45:132
2513599272,10cyclictest28350-21rm21:55:132
25124992714,9cyclictest0-21swapper/120:50:121
25124992714,8cyclictest9147-21ssh22:20:141
25124992713,10cyclictest6171-21ssh21:25:141
25124992712,11cyclictest14276-21apt-get22:05:151
2511599273,14cyclictest20007-21sh23:55:120
25115992716,7cyclictest24494-1kworker/u9:122:15:120
25115992710,13cyclictest12-21ksoftirqd/022:10:120
2514799265,13cyclictest35966irq/58-eth000:15:123
2514799262,6cyclictest0-21swapper/300:30:153
2514799261,6cyclictest0-21swapper/323:25:323
2514799261,11cyclictest0-21swapper/323:55:203
2513599263,15cyclictest27346irq/40-dwc2_hso23:25:132
2513599262,20cyclictest0-21swapper/222:55:142
2512499263,8cyclictest22398-21ssh00:30:231
25124992616,7cyclictest29417-21ssh21:20:141
2512499260,18cyclictest0-21swapper/123:35:121
2511599268,10cyclictest0-21swapper/020:35:110
2511599264,17cyclictest14931-21ssh21:10:280
2511599263,16cyclictest19268-21ssh21:50:130
2511599262,18cyclictest16506-21ssh00:27:430
2511599261,19cyclictest6172-21ssh21:25:140
2514799251,6cyclictest0-21swapper/320:05:013
2514799251,4cyclictest0-21swapper/323:48:523
2514799251,4cyclictest0-21swapper/323:48:523
2514799251,17cyclictest0-21swapper/323:35:133
2513599253,12cyclictest27346irq/40-dwc2_hso20:15:122
2513599251,20cyclictest4571-21latency21:00:232
2512499257,8cyclictest32475-21ssh22:15:121
2512499252,16cyclictest18230-21ssh22:25:141
25124992519,3cyclictest12047-21apt-get19:30:141
25124992513,8cyclictest0-21swapper/121:15:131
2511599257,12cyclictest28341-21diskmemload21:55:120
2511599251,18cyclictest22130-21sh22:45:140
2514799246,5cyclictest0-21swapper/322:05:213
2514799243,8cyclictest0-21swapper/323:05:063
2514799242,6cyclictest0-21swapper/323:30:313
2514799241,7cyclictest0-21swapper/321:30:183
2514799241,6cyclictest0-21swapper/322:45:193
2514799241,6cyclictest0-21swapper/320:40:163
2514799241,6cyclictest0-21swapper/320:10:283
2514799241,6cyclictest0-21swapper/319:15:243
2514799241,6cyclictest0-21swapper/300:25:163
2514799241,5cyclictest0-21swapper/322:40:313
25147992413,9cyclictest33-21ksoftirqd/321:25:143
25147992411,9cyclictest35966irq/58-eth023:10:133
2513599243,16cyclictest27346irq/40-dwc2_hso23:45:152
2513599243,16cyclictest27346irq/40-dwc2_hso23:45:152
2513599242,6cyclictest4999-21grep21:40:282
2513599241,8cyclictest2805466ssh21:35:252
2512499247,13cyclictest14400-21sed23:50:241
2512499247,13cyclictest14400-21sed23:50:241
2512499241,18cyclictest3982-21ssh22:35:141
25115992418,2cyclictest0-21swapper/022:40:140
2511599241,2cyclictest131rcu_preempt21:05:120
25115992411,8cyclictest121ksoftirqd/023:10:130
2511599240,16cyclictest27337-21ssh22:30:130
2511599240,16cyclictest27337-21ssh22:30:130
2514799232,6cyclictest0-21swapper/322:30:173
2514799232,6cyclictest0-21swapper/322:30:163
2514799232,4cyclictest0-21swapper/319:40:013
2514799232,4cyclictest0-21swapper/319:40:013
2514799231,7cyclictest0-21swapper/322:35:223
2514799231,7cyclictest0-21swapper/322:20:223
2514799231,7cyclictest0-21swapper/322:00:253
2514799231,7cyclictest0-21swapper/320:25:153
2514799231,6cyclictest0-21swapper/323:00:233
2514799231,6cyclictest0-21swapper/322:50:213
2514799231,6cyclictest0-21swapper/321:00:173
2514799231,6cyclictest0-21swapper/320:55:223
2514799231,6cyclictest0-21swapper/320:50:223
2514799231,6cyclictest0-21swapper/320:22:213
2514799231,6cyclictest0-21swapper/319:55:173
2514799231,6cyclictest0-21swapper/319:50:163
2514799231,6cyclictest0-21swapper/319:40:163
2514799231,6cyclictest0-21swapper/300:05:293
2514799231,5cyclictest0-21swapper/300:20:273
2513599233,16cyclictest27346irq/40-dwc2_hso19:25:152
2513599233,13cyclictest27346irq/40-dwc2_hso23:50:142
2513599233,13cyclictest27346irq/40-dwc2_hso23:50:132
2513599232,17cyclictest27346irq/40-dwc2_hso22:25:132
2513599232,16cyclictest6128-21acpi21:25:122
2513599232,15cyclictest27346irq/40-dwc2_hso22:40:132
2513599232,11cyclictest10588-21kthreadcore20:35:182
25135992315,4cyclictest261rcuc/222:00:132
25135992311,9cyclictest0-21swapper/220:10:232
2513599231,13cyclictest22121-21ssh22:45:122
2513599231,12cyclictest11721-21ssh22:20:232
2512499238,11cyclictest28410-21ls21:55:131
2512499233,3cyclictest0-21swapper/123:26:251
2512499233,10cyclictest664-21ntpd21:05:131
25124992318,2cyclictest1050-21apt-get21:40:121
25124992313,6cyclictest7237-21apt-get19:25:151
2512499231,19cyclictest30093-21ls19:15:151
2512499231,19cyclictest27335-21sh22:30:131
2512499231,19cyclictest27335-21sh22:30:131
2511599231,19cyclictest9139-21ssh22:20:130
2511599230,19cyclictest17009-21cut19:35:170
2511599230,19cyclictest17009-21cut19:35:170
2514799222,7cyclictest0-21swapper/300:35:153
2514799222,5cyclictest0-21swapper/323:20:193
2514799221,6cyclictest0-21swapper/321:15:173
2514799221,6cyclictest0-21swapper/320:45:163
2514799221,6cyclictest0-21swapper/320:30:293
2514799221,6cyclictest0-21swapper/320:05:283
2514799221,6cyclictest0-21swapper/319:45:213
2514799221,6cyclictest0-21swapper/319:25:223
2513599223,10cyclictest16588-21sed23:35:182
2513599222,15cyclictest0-21swapper/223:10:122
2513599221,12cyclictest2144-21wc19:50:232
25135992210,9cyclictest25387-1kworker/u9:222:30:152
25135992210,9cyclictest25387-1kworker/u9:222:30:152
25135992210,5cyclictest6109-21ssh22:16:562
25135992210,5cyclictest28379-21ssh22:10:392
2512499228,11cyclictest32725-21fschecks_time20:25:161
2512499222,4cyclictest21-21ksoftirqd/100:03:451
2512499221,16cyclictest18154-21ls20:40:271
2511599221,17cyclictest5778-21ls00:05:150
2514799211,6cyclictest0-21swapper/320:35:143
2514799211,6cyclictest0-21swapper/319:20:153
2514799211,13cyclictest35966irq/58-eth019:10:123
2513599217,7cyclictest27-21ksoftirqd/221:16:412
2513599216,7cyclictest2467-21ntp_states22:50:272
2513599213,9cyclictest8182-21basename20:00:012
2513599213,15cyclictest27903-21grep20:50:282
2513599213,14cyclictest27346irq/40-dwc2_hso19:20:132
2513599212,10cyclictest16077-21kthreadcore21:30:212
25135992115,3cyclictest27-21ksoftirqd/221:05:132
2513599211,10cyclictest7423-21irqcore23:30:192
2512499219,9cyclictest26456-21munin-node19:45:121
2512499218,10cyclictest10225-21sed00:25:171
2512499217,11cyclictest32575-21sed20:55:281
2512499217,10cyclictest2366366ssh23:55:251
2512499214,3cyclictest21-21ksoftirqd/123:21:201
25124992111,7cyclictest5681-21apt-get00:05:151
25124992110,8cyclictest31117-21awk20:20:251
2512499210,8cyclictest0-21swapper/122:50:211
2512499210,18cyclictest28500-21ls19:10:211
2511599218,10cyclictest31480-21wc19:50:170
2511599211,16cyclictest14303-21ssh22:05:150
2511599211,15cyclictest11763-21ssh23:15:050
2513599207,10cyclictest30217-21fschecks_time19:15:182
2513599203,7cyclictest28938-21apt-get20:55:122
2513599202,15cyclictest19448-21fschecks_count20:45:172
2513599201,8cyclictest930-21snmpd21:10:042
2513599201,11cyclictest29143-21ssh00:00:142
2513599201,10cyclictest9071-21sed20:00:202
2513599201,10cyclictest8147-21wc20:30:222
2513599201,10cyclictest29799-21missed_timers19:45:242
2512499209,9cyclictest31661-21ntp_states00:35:271
25124992014,3cyclictest8419-21apt-get20:00:161
2512499200,17cyclictest22839-21find20:15:121
2511599208,3cyclictest131rcu_preempt21:35:300
2511599202,3cyclictest131rcu_preempt19:10:110
25115992013,2cyclictest0-21swapper/023:50:120
25115992013,2cyclictest0-21swapper/023:50:120
2511599201,14cyclictest7913-21timerandwakeup22:35:290
2511599200,15cyclictest32237-21latency_hist20:25:010
2513599191,8cyclictest21983-21irqcore19:40:182
2513599191,10cyclictest14919-21irqcore20:40:172
2513599190,11cyclictest28331-21kthreadcore20:20:202
2512499198,8cyclictest25495-21unixbench_multi19:40:291
2512499192,14cyclictest2659-21cut21:00:211
2512499192,14cyclictest14941-21tr20:05:211
25124991915,2cyclictest16854-21apt-get19:35:131
25124991915,2cyclictest16854-21apt-get19:35:131
2512499191,15cyclictest3619-21find19:55:111
2511599199,7cyclictest14026-21wc22:05:010
2511599191,16cyclictest29810-21irqrtprio00:00:190
2511599191,15cyclictest701-21ssh21:20:260
2511599191,13cyclictest2082-21ssh23:45:180
2511599191,13cyclictest2082-21ssh23:45:170
2513599181,10cyclictest25207-21latency_hist19:10:022
2512499186,9cyclictest8184-21cat20:30:221
25115991812,4cyclictest18977-21kthreadcore22:25:210
2511599181,14cyclictest3628-21apt-get19:55:110
2511599181,14cyclictest15969-21kthreadcore20:40:190
2511599181,13cyclictest27452-21cat00:15:220
25115991810,6cyclictest5116-21cat21:00:290
25115991810,5cyclictest5568-21tr20:30:190
2511599173,11cyclictest27847-21ssh23:05:210
2511599172,6cyclictest7200-21cat23:30:180
2511599172,10cyclictest0-21swapper/020:00:160
2511599171,8cyclictest19293-21/usr/sbin/munin20:45:170
2511599171,11cyclictest30671-21tr19:15:210
25115991710,5cyclictest28685-21sshd20:51:440
25115991710,4cyclictest28330-21tail19:45:210
2511599166,7cyclictest4867-21kthreadcore19:20:210
2511599162,10cyclictest0-21swapper/020:10:140
2511599161,3cyclictest0-21swapper/020:15:070
25115991611,3cyclictest13230-21apt-get20:05:130
2511599161,12cyclictest12187-21usage_Anybus22:55:310
2512499152,9cyclictest25160-21latency_hist19:10:021
2511599157,5cyclictest25142-21wc19:10:020
2511599151,10cyclictest13588-21kthreadcore19:30:200
251479991,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional