You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 06:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Fri May 01, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2543399723,62cyclictest4853-21ssh22:00:121
25433997122,8cyclictest21-21ksoftirqd/119:40:121
25447996038,11cyclictest35966irq/58-eth023:10:153
2543399603,47cyclictest29733-21ssh21:20:121
25447995631,12cyclictest35966irq/58-eth021:25:143
2543399562,38cyclictest7727-21ssh00:10:091
25438995518,33cyclictest131rcu_preempt00:30:132
25433995522,22cyclictest21-21ksoftirqd/119:15:121
25447995426,17cyclictest35966irq/58-eth021:20:133
2542599531,38cyclictest30143-21ssh22:50:120
2542599521,44cyclictest10063-21ssh21:45:110
2543899504,31cyclictest8836-21ssh23:50:142
2543899501,31cyclictest1427-21ssh23:10:142
2543399504,6cyclictest131rcu_preempt23:20:121
2542599502,38cyclictest15823-21ssh23:00:140
2543399494,7cyclictest131rcu_preempt21:00:131
2544799453,24cyclictest35966irq/58-eth022:40:133
2544799443,25cyclictest35966irq/58-eth023:20:143
25447994423,11cyclictest35966irq/58-eth021:40:113
2543399447,25cyclictest8547-21ssh22:20:131
2543399442,37cyclictest12138-21ssh22:40:141
25425994328,9cyclictest24561-21apt-get20:50:140
25425994327,10cyclictest523-21apt-get20:25:130
2543399426,22cyclictest753-21sendmail-mta20:55:141
25425994233,5cyclictest7717-21apt-get00:10:090
2542599422,28cyclictest14201-21rm23:35:140
25425994221,14cyclictest0-21swapper/000:35:120
2544799412,24cyclictest35966irq/58-eth023:55:123
2543899411,27cyclictest29739-21ssh21:20:132
25425994128,8cyclictest0-21swapper/023:25:110
2543399406,27cyclictest11263-21diskmemload21:40:121
2543399404,25cyclictest24836-21ssh23:05:161
2542599404,26cyclictest6310-21ssh21:25:140
2544799362,21cyclictest35966irq/58-eth021:10:133
2542599367,23cyclictest12-21ksoftirqd/021:35:120
25425993628,3cyclictest13910-21apt-get22:05:130
2544799353,18cyclictest0-21swapper/322:00:133
2542599357,22cyclictest21170-21ssh22:45:130
25425993528,4cyclictest29414-21apt-get20:55:120
25425993528,3cyclictest1405-21apt-get23:10:140
25425993525,6cyclictest26927-21apt-get19:45:120
2542599348,16cyclictest664-21ntpd21:30:140
2542599345,9cyclictest9772-21users22:00:320
25425993426,5cyclictest24809-21apt-get23:05:150
25425993424,5cyclictest30466-21apt-get19:15:120
2543899330,28cyclictest616-21systemd-logind23:06:482
25425993326,4cyclictest22107-21apt-get19:40:130
2542599331,27cyclictest11655-21ssh21:10:120
2543899325,13cyclictest930-21snmpd19:45:122
2543899322,8cyclictest0-21swapper/222:57:522
25425993225,3cyclictest19467-21apt-get23:20:130
2542599322,23cyclictest12123-21ssh22:40:120
2544799313,21cyclictest35966irq/58-eth023:05:153
2544799312,12cyclictest35966irq/58-eth022:50:123
2543899312,8cyclictest0-21swapper/200:10:092
2543899311,18cyclictest959-21rm21:40:132
25425993125,3cyclictest1818-21apt-get21:00:140
25425993125,2cyclictest20649-21apt-get21:15:140
25425993119,8cyclictest17278-21apt-get19:35:110
25425993117,11cyclictest0-21swapper/000:00:130
25447993014,10cyclictest35966irq/58-eth000:25:163
2543399303,21cyclictest17806-21ssh23:55:141
2542599307,15cyclictest0-21swapper/022:55:130
2542599305,10cyclictest24339-21rm23:57:360
25425993015,11cyclictest0-21swapper/022:20:110
2544799295,13cyclictest35966irq/58-eth023:50:143
25447992916,9cyclictest35966irq/58-eth022:45:143
2543899292,21cyclictest2898-21ssh23:27:482
2543899291,21cyclictest0-21swapper/221:05:142
2544799284,16cyclictest0-21swapper/321:30:133
2544799281,17cyclictest35966irq/58-eth022:15:133
2544799281,17cyclictest35966irq/58-eth000:30:143
2542599287,9cyclictest1128-21ssh00:23:580
2542599281,19cyclictest23282-21ssh23:40:170
2544799274,13cyclictest35966irq/58-eth022:10:123
25447992713,11cyclictest0-21swapper/323:00:143
2543899274,15cyclictest27346irq/40-dwc2_hso20:55:122
2543899272,15cyclictest27346irq/40-dwc2_hso20:00:132
2543399273,18cyclictest2512-21ssh00:25:161
2542599274,9cyclictest11263-21diskmemload22:15:220
2542599273,8cyclictest316-21ssh22:32:180
25425992721,3cyclictest8825-21apt-get23:50:160
25425992718,3cyclictest11370-21apt-get00:30:130
25425992714,8cyclictest22924-21ssh22:10:120
2544799263,16cyclictest0-21swapper/323:27:473
2544799262,19cyclictest35966irq/58-eth021:35:123
2544799262,17cyclictest0-21swapper/322:56:473
2543899263,7cyclictest0-21swapper/223:17:532
2543899262,9cyclictest0-21swapper/220:25:132
2543899262,15cyclictest27346irq/40-dwc2_hso19:35:112
2543899261,9cyclictest20660-21ssh21:15:142
2543899260,6cyclictest11263-21diskmemload22:55:022
2542599269,12cyclictest18040-21sed22:25:170
2542599268,9cyclictest0-21swapper/020:30:130
2542599268,8cyclictest0-21swapper/023:45:130
2542599265,17cyclictest12-21ksoftirqd/020:45:130
2542599265,10cyclictest7233-21irqrtprio21:05:190
2542599264,9cyclictest4553-21ssh00:25:240
2542599264,7cyclictest0-21swapper/023:16:060
2542599263,16cyclictest0-21swapper/019:10:120
25425992620,3cyclictest8882-21apt-get20:00:120
25425992611,11cyclictest29721-21apt-get21:20:140
2544799256,9cyclictest35966irq/58-eth023:45:143
2543899252,18cyclictest6593-21ssh23:47:472
2543899251,9cyclictest17692-21ssh21:48:472
2543899251,12cyclictest4914-21cpuspeed22:00:132
2543899250,10cyclictest21185-21ssh22:45:152
2542599254,9cyclictest6599-21ssh00:09:100
2542599253,9cyclictest0-21swapper/022:36:500
2542599251,8cyclictest0-21swapper/000:16:070
25425992513,8cyclictest10777-21/usr/sbin/munin20:35:160
25425992513,8cyclictest10777-21/usr/sbin/munin20:35:160
2542599251,17cyclictest12-21ksoftirqd/020:40:110
2544799244,16cyclictest35966irq/58-eth023:30:153
2544799241,15cyclictest0-21swapper/300:35:133
2543899242,18cyclictest0-21swapper/223:20:132
2543899241,8cyclictest3323-21ssh21:58:472
2543899241,18cyclictest27583-21ssh21:54:482
2543899241,17cyclictest20339-21ssh00:35:132
2543899241,16cyclictest13114-21kthreadcore19:30:182
2543399247,5cyclictest26484-21kworker/u8:023:45:191
2543399246,14cyclictest0-21swapper/119:45:141
2543399243,9cyclictest131rcu_preempt00:15:131
2542599247,13cyclictest29310-21kthreadcore20:20:180
25425992410,10cyclictest23434-21/usr/sbin/munin20:15:160
2544799235,10cyclictest0-21swapper/322:30:153
2544799232,14cyclictest0-21swapper/322:20:123
2544799230,18cyclictest35966irq/58-eth021:05:143
2543899232,16cyclictest0-21swapper/223:00:142
2543899232,16cyclictest0-21swapper/222:40:132
2543899232,16cyclictest0-21swapper/200:07:472
2543899232,16cyclictest0-21swapper/200:02:582
2543899232,14cyclictest0-21swapper/219:15:132
2543899231,8cyclictest29944-21unixbench_singl00:20:292
2543399233,14cyclictest8851-21ssh23:50:151
25433992314,6cyclictest20665-21ssh21:15:151
25433992312,8cyclictest25009-21sshd21:51:471
2543399231,18cyclictest5374-21apt-get20:30:131
2543399231,18cyclictest1430-21sh23:10:141
25425992317,3cyclictest4082-21apt-get19:55:130
25425992311,9cyclictest22020-21open_inodes20:10:290
2544799225,11cyclictest0-21swapper/321:55:153
2543899226,11cyclictest27-21ksoftirqd/220:50:122
2543899222,9cyclictest131rcu_preempt23:30:302
2543899222,15cyclictest0-21swapper/221:34:472
25438992216,4cyclictest24546irq/40-fe98000000:25:162
25438992211,9cyclictest0-21swapper/220:40:152
2543899221,16cyclictest27346irq/40-dwc2_hso23:55:132
2543899221,13cyclictest0-21swapper/219:40:132
2543899220,15cyclictest11755-21ls22:40:022
25433992216,3cyclictest27932-21sh21:35:241
2543399221,19cyclictest167-21systemd-network22:19:521
2542599229,9cyclictest0-21swapper/019:20:110
2542599228,8cyclictest0-21swapper/021:50:120
2542599227,12cyclictest0-21swapper/019:30:120
2542599220,7cyclictest0-21swapper/021:57:110
2544799219,8cyclictest35966irq/58-eth000:00:133
2544799216,8cyclictest0-21swapper/323:15:143
2543899212,3cyclictest12402-21grep21:10:172
2543899212,17cyclictest0-21swapper/219:20:132
2543899212,12cyclictest0-21swapper/219:50:122
25438992114,4cyclictest9886-21ssh21:25:262
2543899211,3cyclictest26898-21cat22:10:292
2543899211,16cyclictest930-21snmpd20:20:182
2543899211,13cyclictest32127-21cut22:15:162
2543899211,12cyclictest25484-21ls19:10:022
2543899211,11cyclictest26781-21ssh22:30:182
2543899211,11cyclictest19812-21kthreadcore00:15:232
2543399212,16cyclictest13918-21ssh22:05:121
2543399211,8cyclictest6573-21latency_hist22:55:031
25433992116,3cyclictest26518-21ssh22:30:151
25433992113,6cyclictest6751-21ssh22:55:131
25433992113,6cyclictest17483-21ssh23:35:231
25433992111,8cyclictest12267-21ssh23:15:201
2543399211,17cyclictest17493-21ssh22:25:121
25433992110,7cyclictest28862-21latency_hist00:40:011
2542599217,11cyclictest4689-21tr21:40:260
2542599213,4cyclictest12-21ksoftirqd/023:30:520
25425992110,8cyclictest31806-21entropy19:50:140
2544799203,12cyclictest0-21swapper/300:07:593
2544799202,11cyclictest0-21swapper/320:45:133
2544799201,14cyclictest0-21swapper/322:25:133
2543899201,9cyclictest16902-21kthreadcore22:05:212
2543899201,9cyclictest16771-21kthreadcore23:35:222
2543899201,5cyclictest26958-21sh23:40:282
2543899201,11cyclictest23450-21fschecks_count20:15:152
2543899201,10cyclictest26582-21cut21:35:192
2543899201,10cyclictest11767-21kthreadcore22:20:222
2543399209,3cyclictest131rcu_preempt22:45:201
2543399208,3cyclictest131rcu_preempt20:15:001
2543399207,3cyclictest131rcu_preempt23:40:231
2543399201,15cyclictest28472-21ssh21:55:171
2543399201,14cyclictest11380-21ssh00:30:131
25433992011,3cyclictest11263-21diskmemload22:39:581
2543399201,13cyclictest10337-21ls21:45:161
25433992010,6cyclictest29518-21ssh00:20:271
25425992011,6cyclictest16974-21memory20:05:210
2544799197,6cyclictest0-21swapper/321:15:313
25447991912,4cyclictest0-21swapper/300:20:183
2544799191,13cyclictest0-21swapper/323:35:143
2544799191,11cyclictest0-21swapper/320:05:163
2543899195,11cyclictest20253-21grep20:45:182
2543899194,6cyclictest27-21ksoftirqd/220:35:202
2543899194,6cyclictest27-21ksoftirqd/220:35:192
2543899192,15cyclictest18822-21kthreadcore22:25:202
2543899191,9cyclictest1821-21cpu21:00:142
2543899191,8cyclictest27993-21kthreadcore19:10:202
2543899191,15cyclictest12207-21cron19:30:012
2543899191,15cyclictest12207-21cron19:30:002
2543899191,14cyclictest5504-21fschecks_count20:30:172
2543899191,10cyclictest18773-21sed20:10:202
2543899191,10cyclictest14519-21kthreadcore20:05:172
2543399193,2cyclictest131rcu_preempt22:10:121
2543399191,9cyclictest31334-21kthreadcore23:25:201
25433991911,5cyclictest19145-21ssh23:00:221
25433991911,5cyclictest15946-21usage_Anybus21:10:291
2543399191,14cyclictest25572-21apt-get19:10:131
2543399191,11cyclictest26736-21find00:00:121
2542599193,13cyclictest7815-21cut19:25:160
2542599193,13cyclictest7815-21cut19:25:160
2544799189,6cyclictest0-21swapper/300:17:153
2544799188,6cyclictest0-21swapper/300:10:093
2544799183,10cyclictest0-21swapper/321:52:413
2544799182,10cyclictest0-21swapper/320:55:043
2544799182,10cyclictest0-21swapper/320:40:133
2544799182,10cyclictest0-21swapper/320:15:143
2544799182,10cyclictest0-21swapper/320:00:143
2544799182,10cyclictest0-21swapper/319:50:133
2544799182,10cyclictest0-21swapper/319:40:133
2544799182,10cyclictest0-21swapper/319:35:123
2544799182,10cyclictest0-21swapper/319:30:133
2544799182,10cyclictest0-21swapper/319:15:043
25447991812,4cyclictest0-21swapper/321:45:223
2544799181,13cyclictest0-21swapper/320:20:273
2543899181,8cyclictest4085-21cpu19:55:132
2543399182,6cyclictest6418-21df_inode21:25:161
2543399181,9cyclictest3576-21kthreadcore20:25:201
2543399181,8cyclictest27933-21munin-run20:20:011
25433991811,4cyclictest8290-21kthreadcore23:30:221
25433991811,4cyclictest11263-21diskmemload21:30:141
2544799179,6cyclictest0-21swapper/322:37:203
2544799179,6cyclictest0-21swapper/322:06:303
2544799172,10cyclictest0-21swapper/323:40:293
2544799172,10cyclictest0-21swapper/319:45:133
2544799172,10cyclictest0-21swapper/319:20:133
2544799172,10cyclictest0-21swapper/319:10:133
2544799171,10cyclictest0-21swapper/321:00:153
2543399179,6cyclictest5582-21sshd00:07:581
2543399176,9cyclictest32755-21cron20:25:011
2543399172,10cyclictest0-21swapper/120:00:141
2543399171,8cyclictest19524-21kthreadcore19:35:201
2543399171,7cyclictest11349-21latency_hist21:10:021
25433991711,4cyclictest0-21swapper/119:50:301
25433991710,5cyclictest0-21swapper/120:05:211
2544799162,9cyclictest0-21swapper/320:10:133
2544799162,9cyclictest0-21swapper/319:25:143
2544799162,9cyclictest0-21swapper/319:25:143
2544799162,6cyclictest0-21swapper/320:25:123
2544799162,10cyclictest0-21swapper/319:55:133
2544799161,12cyclictest0-21swapper/320:30:123
2543399163,10cyclictest1-21systemd20:52:291
2543399162,11cyclictest5515-21kthreadcore19:55:201
2543399162,11cyclictest5175-21kthreadcore19:20:211
2543399162,11cyclictest12748-21kthreadcore20:35:201
2543399162,11cyclictest12748-21kthreadcore20:35:191
2543399161,13cyclictest22882-21kthreadcore20:45:211
2543399161,13cyclictest12964-21irqrtprio19:30:171
2543399161,12cyclictest14969-21apt-get20:40:121
2543399160,12cyclictest8954-21cut19:25:201
2543399160,12cyclictest8954-21cut19:25:191
2542599164,10cyclictest25441-21latency_hist19:10:010
2544799151,7cyclictest0-21swapper/320:55:033
2544799151,7cyclictest0-21swapper/320:40:023
2544799151,7cyclictest0-21swapper/320:40:013
2543399131,9cyclictest25482-21latency_hist19:10:011
2544799111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional