You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 11:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Jun 19, 2026 00:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19373996226,24cyclictest35966irq/58-eth021:20:123
19336995236,10cyclictest18095-21ssh00:15:120
1933699512,7cyclictest131rcu_preempt20:05:140
19373995021,16cyclictest35966irq/58-eth023:10:143
19373994938,6cyclictest35966irq/58-eth023:15:133
19348994810,29cyclictest0-21swapper/120:20:111
1934899462,31cyclictest8987-21ssh00:10:121
1934899461,35cyclictest23629-21ssh21:20:111
1934899431,31cyclictest18769-21ssh21:35:121
1933699432,33cyclictest23250-21ssh00:00:150
1933699431,34cyclictest12768-21ssh00:30:130
19348994128,9cyclictest31032-21apt-get22:35:141
1933699401,30cyclictest14155-21ssh23:55:130
1933699401,30cyclictest14155-21ssh23:55:130
1937399394,25cyclictest35966irq/58-eth021:15:133
1937399394,22cyclictest35966irq/58-eth022:40:123
19348993919,13cyclictest14454-21ssh21:15:141
1934899391,27cyclictest7729-21ssh22:40:121
1933699392,28cyclictest10323-21ssh23:35:130
1933699391,34cyclictest11709-21ssh23:00:130
1933699391,28cyclictest2614-21ssh22:55:130
1933699391,28cyclictest2614-21ssh22:55:130
1937399383,27cyclictest35966irq/58-eth022:50:123
19348993831,3cyclictest7422-21apt-get20:05:131
1934899381,32cyclictest29895-21ssh23:10:131
19348993810,10cyclictest32659-21ssh22:16:291
1933699382,26cyclictest8715-21ssh22:05:110
19336993813,12cyclictest131rcu_preempt20:35:130
19361993724,9cyclictest3830-21apt-get20:35:122
19348993721,10cyclictest25913-21apt-get22:50:121
1937399365,19cyclictest35966irq/58-eth022:20:133
19348993628,4cyclictest24272-21apt-get19:15:121
19348993616,17cyclictest0-21swapper/120:50:131
19373993519,9cyclictest35966irq/58-eth000:35:143
1936199356,15cyclictest27346irq/40-dwc2_hso19:40:112
1934899352,27cyclictest24614-21ssh23:25:131
1933699352,23cyclictest6501-21ssh23:15:120
1933699351,25cyclictest31979-21sh22:00:130
19361993423,6cyclictest11690-21apt-get23:00:112
19348993421,10cyclictest3695-21apt-get22:20:131
1937399332,21cyclictest35966irq/58-eth022:45:133
19361993314,16cyclictest0-21swapper/219:10:112
19348993326,3cyclictest31963-21apt-get22:00:121
19348993318,9cyclictest468-21ssh21:25:151
1933699338,17cyclictest12-21ksoftirqd/020:30:130
1933699332,25cyclictest31058-21ssh22:35:140
1937399323,20cyclictest35966irq/58-eth023:45:133
1937399323,17cyclictest35966irq/58-eth021:25:153
19361993224,3cyclictest6272-21apt-get19:30:122
1936199321,15cyclictest7710-21ssh22:40:122
1934899329,7cyclictest28681-21ssh21:56:391
1934899327,8cyclictest25562-21ntp_192.168.11422:30:251
1934899327,12cyclictest11802-21ssh23:00:141
1934899326,11cyclictest18382-21cut22:10:171
19348993227,2cyclictest31469-21apt-get20:30:121
19373993114,13cyclictest35966irq/58-eth021:45:143
1937399311,20cyclictest35966irq/58-eth019:40:123
1937399311,15cyclictest35966irq/58-eth019:55:123
1936199316,20cyclictest0-21swapper/219:45:112
1936199314,15cyclictest27346irq/40-dwc2_hso20:15:112
19361993115,2cyclictest0-21swapper/221:45:142
19361993112,13cyclictest27-21ksoftirqd/223:31:402
1936199311,18cyclictest29899-21ssh23:10:142
1934899318,8cyclictest30469-21ssh00:03:201
1934899318,8cyclictest25915-21ssh23:05:451
1934899318,8cyclictest16962-21ssh22:09:271
1934899318,10cyclictest6891-21ssh23:31:241
1934899317,9cyclictest16643-21kthreadcore23:55:221
1934899317,9cyclictest16643-21kthreadcore23:55:221
1934899317,9cyclictest13326-21ssh21:14:111
1934899317,11cyclictest31952-21cat00:20:321
1934899317,10cyclictest4471-21ssh21:45:021
1934899317,10cyclictest23532-21sh23:24:121
1934899314,11cyclictest25031-21usage_EthcTask19:45:281
19348993121,5cyclictest3752-21apt-get00:25:151
1937399304,17cyclictest35966irq/58-eth023:55:143
1937399304,17cyclictest35966irq/58-eth023:55:133
1936199308,6cyclictest126601ntpq00:10:252
1936199303,17cyclictest0-21swapper/221:15:142
1936199301,3cyclictest27346irq/40-dwc2_hso22:00:122
1936199301,16cyclictest167-21systemd-network20:05:142
1934899307,8cyclictest27955-21sh00:37:021
1934899307,10cyclictest17269-21cat22:25:301
1934899305,9cyclictest20558-21ntp_states22:45:271
1934899303,22cyclictest32331-21ssh00:05:141
1933699302,5cyclictest131rcu_preempt22:30:140
1933699302,22cyclictest21860-21ssh00:35:140
1937399292,17cyclictest35966irq/58-eth023:35:133
1936199293,15cyclictest27-21ksoftirqd/223:46:502
1936199291,6cyclictest27-21ksoftirqd/221:30:182
1936199291,17cyclictest27346irq/40-dwc2_hso23:25:122
1936199291,16cyclictest27346irq/40-dwc2_hso22:55:142
1936199291,16cyclictest27346irq/40-dwc2_hso22:55:132
1934899298,7cyclictest11194-21sh22:59:541
1934899298,7cyclictest11194-21sh22:59:531
1934899296,8cyclictest4912-21diskmemload23:35:381
1934899296,8cyclictest29972-21users19:50:291
1934899296,10cyclictest21531-21cut19:10:201
1934899293,11cyclictest0-21swapper/120:00:111
1937399283,16cyclictest35966irq/58-eth023:05:133
1936199289,10cyclictest131rcu_preempt23:55:262
1936199289,10cyclictest131rcu_preempt23:55:252
1936199283,21cyclictest0-21swapper/221:05:132
1936199283,21cyclictest0-21swapper/221:05:122
1936199283,14cyclictest27346irq/40-dwc2_hso20:20:122
1936199283,14cyclictest27-21ksoftirqd/221:38:002
1936199282,11cyclictest32049-21usage_Anybus21:00:302
19361992814,11cyclictest21828-21apt-get00:35:142
1936199281,17cyclictest19412-21ssh23:40:142
1934899287,8cyclictest4912-21diskmemload21:34:441
1934899286,8cyclictest26694-21sendmail_mailqu20:55:281
1934899286,10cyclictest4167-21unixbench_multi21:05:291
1934899286,10cyclictest4167-21unixbench_multi21:05:291
1934899285,8cyclictest4912-21diskmemload23:19:271
19348992821,3cyclictest12756-21apt-get00:30:111
19348992813,13cyclictest5087-21apt-get23:50:121
19348992813,13cyclictest5087-21apt-get23:50:121
1936199275,17cyclictest0-21swapper/222:35:132
1936199275,15cyclictest27346irq/40-dwc2_hso19:55:132
1936199274,5cyclictest271ksoftirqd/200:34:262
1936199273,9cyclictest0-21swapper/222:05:122
1936199273,6cyclictest25061-21kthreadcore20:55:202
1936199273,4cyclictest271ksoftirqd/221:25:182
1936199273,15cyclictest27-21ksoftirqd/200:15:132
1936199272,19cyclictest27149-21ssh00:20:152
1936199272,15cyclictest27346irq/40-dwc2_hso00:05:142
1936199272,13cyclictest27-21ksoftirqd/222:20:172
1934899276,7cyclictest0-21swapper/100:18:481
1934899274,8cyclictest0-21swapper/120:35:311
19348992710,14cyclictest0-21swapper/120:45:141
1937399262,15cyclictest35966irq/58-eth022:05:123
1937399262,12cyclictest35966irq/58-eth021:10:133
1936199265,8cyclictest27-21ksoftirqd/219:50:252
1936199265,18cyclictest0-21swapper/219:15:122
1936199265,14cyclictest0-21swapper/223:07:352
1936199265,13cyclictest0-21swapper/222:15:482
1936199265,11cyclictest131rcu_preempt21:54:352
1936199265,11cyclictest131rcu_preempt21:54:352
1936199264,9cyclictest27-21ksoftirqd/221:10:172
1936199264,5cyclictest27-21ksoftirqd/221:23:252
1936199264,4cyclictest27-21ksoftirqd/200:00:362
1936199263,5cyclictest271ksoftirqd/223:50:442
1936199263,5cyclictest271ksoftirqd/223:50:442
1936199263,4cyclictest271ksoftirqd/222:45:292
1936199263,4cyclictest271ksoftirqd/219:35:332
1936199263,15cyclictest27-21ksoftirqd/200:25:502
1936199262,19cyclictest21927-21ssh22:30:152
1936199262,15cyclictest27346irq/40-dwc2_hso20:45:132
1936199262,13cyclictest27-21ksoftirqd/219:25:262
1934899268,12cyclictest0-21swapper/119:40:131
1934899267,8cyclictest11613-21ssh21:47:561
19348992620,3cyclictest8683-21apt-get20:40:111
19348992610,11cyclictest0-21swapper/119:55:101
1933699264,5cyclictest8977-21apt-get00:10:120
1933699262,18cyclictest20884-21ssh23:05:140
1937399258,11cyclictest35966irq/58-eth021:30:133
1937399252,15cyclictest0-21swapper/322:15:133
1936199254,5cyclictest27-21ksoftirqd/223:15:152
1936199253,4cyclictest27-21ksoftirqd/221:55:282
1936199253,14cyclictest27-21ksoftirqd/222:25:252
1936199253,13cyclictest27-21ksoftirqd/222:10:392
1936199251,17cyclictest0-21swapper/223:20:132
1934899256,8cyclictest91-21pr/ttyS020:20:011
1934899255,7cyclictest0-21swapper/120:10:141
1934899255,7cyclictest0-21swapper/120:10:141
19348992513,9cyclictest19814-21ssh21:51:531
19348992513,9cyclictest19814-21ssh21:51:521
1934899251,16cyclictest19428-21ssh23:40:141
1933699252,18cyclictest1227-21ssh23:30:120
1933699251,19cyclictest13746-21ssh21:50:120
1933699251,19cyclictest13746-21ssh21:50:110
1933699251,18cyclictest5349-21ssh21:10:150
1933699251,13cyclictest4652-21ssh21:45:130
1937399241,14cyclictest35966irq/58-eth023:20:133
1937399241,13cyclictest35966irq/58-eth021:35:123
1936199247,3cyclictest0-21swapper/220:30:152
1936199244,3cyclictest27-21ksoftirqd/222:54:302
1936199243,5cyclictest27-21ksoftirqd/220:40:212
1936199243,13cyclictest27-21ksoftirqd/221:40:312
1936199242,5cyclictest27-21ksoftirqd/219:20:132
1936199242,5cyclictest27-21ksoftirqd/219:20:122
1936199242,14cyclictest27-21ksoftirqd/223:36:572
1936199241,10cyclictest18489-21cut20:50:172
19348992414,7cyclictest28516-21apt-get23:45:131
19348992412,9cyclictest1443-21sshd19:25:101
1937399235,11cyclictest35966irq/58-eth021:55:143
1937399233,15cyclictest0-21swapper/323:30:123
19361992314,7cyclictest0-21swapper/220:00:102
19348992312,9cyclictest0-21swapper/121:00:121
19348992312,8cyclictest13736-21cut19:35:231
19348992311,9cyclictest9744-21uname19:30:271
1933699239,3cyclictest131rcu_preempt22:42:540
1933699232,17cyclictest25930-21ssh22:50:130
1933699231,18cyclictest32524-21cut00:05:160
1933699231,18cyclictest22885-21ssh21:55:130
1933699231,16cyclictest3701-21ssh22:20:130
1933699231,15cyclictest26987-21ssh22:15:110
19336992311,4cyclictest12-21ksoftirqd/022:25:180
19336992310,7cyclictest0-21swapper/023:20:140
1937399224,12cyclictest0-21swapper/300:15:143
1937399222,12cyclictest35966irq/58-eth000:30:123
1933699224,14cyclictest5104-21rm23:50:140
1933699224,14cyclictest5104-21rm23:50:140
1937399212,14cyclictest0-21swapper/321:50:123
1937399212,14cyclictest0-21swapper/321:50:113
1937399211,11cyclictest35966irq/58-eth000:00:143
1936199211,11cyclictest29707-21kthreadcore20:25:212
19348992111,7cyclictest26639-21/usr/sbin/munin20:25:231
1933699211,4cyclictest131rcu_preempt19:55:260
19336992113,6cyclictest4905-21cron21:10:010
19336992113,6cyclictest4905-21cron21:10:000
1933699211,16cyclictest25458-21apt-get19:50:110
1937399209,6cyclictest0-21swapper/300:12:393
1937399202,14cyclictest35966irq/58-eth022:30:133
19373992011,6cyclictest0-21swapper/323:50:233
19373992011,6cyclictest0-21swapper/323:50:223
19373992010,8cyclictest0-21swapper/322:38:413
19373992010,6cyclictest35966irq/58-eth023:40:143
1936199202,4cyclictest27-21ksoftirqd/220:10:282
1936199202,4cyclictest27-21ksoftirqd/220:10:272
19348992011,6cyclictest1208-21sshd19:21:101
19348992011,6cyclictest1208-21sshd19:21:101
1933699209,3cyclictest0-21swapper/023:40:280
1933699207,4cyclictest131rcu_preempt23:15:010
1933699207,4cyclictest131rcu_preempt19:40:170
1933699203,10cyclictest16834-21ssh22:45:130
1933699202,3cyclictest131rcu_preempt22:10:300
1933699202,15cyclictest6217-21latency_hist19:30:010
19336992013,5cyclictest0-21swapper/021:20:240
1933699201,2cyclictest131rcu_preempt21:25:160
19336992012,3cyclictest0-21swapper/019:15:120
19336992011,6cyclictest22495-21sh21:19:110
1933699201,10cyclictest13453-21cat20:45:020
19336992010,8cyclictest19489-21apt-get19:10:120
1933699200,5cyclictest18928-21fschecks_count21:35:160
1937399199,3cyclictest0-21swapper/322:27:593
1937399194,10cyclictest0-21swapper/322:55:133
1937399194,10cyclictest0-21swapper/322:55:123
1937399192,11cyclictest0-21swapper/319:50:123
1937399191,12cyclictest0-21swapper/320:50:123
1937399191,12cyclictest0-21swapper/320:15:153
1933699197,3cyclictest12-21ksoftirqd/000:27:020
1933699192,2cyclictest131rcu_preempt21:30:560
1933699192,13cyclictest28533-21ssh23:45:140
1933699191,8cyclictest131rcu_preempt19:35:260
1933699191,2cyclictest131rcu_preempt00:20:270
1933699190,14cyclictest31765-21switchtime21:40:280
1937399189,6cyclictest35966irq/58-eth000:05:133
1937399189,5cyclictest0-21swapper/321:40:193
1937399182,8cyclictest0-21swapper/322:00:153
1937399182,10cyclictest0-21swapper/320:45:153
1937399182,10cyclictest0-21swapper/320:40:113
1937399182,10cyclictest0-21swapper/320:20:123
1937399182,10cyclictest0-21swapper/319:35:133
19373991812,4cyclictest0-21swapper/300:25:193
1937399181,13cyclictest0-21swapper/319:30:283
19373991810,6cyclictest0-21swapper/323:02:123
1933699185,10cyclictest12565-21irqcore20:10:170
1933699185,10cyclictest12565-21irqcore20:10:160
19336991813,3cyclictest0-21swapper/020:00:150
19336991812,4cyclictest0-21swapper/019:45:150
1933699181,13cyclictest13233-21apt-get20:45:030
19336991810,6cyclictest27714-21sshd20:58:110
1937399179,6cyclictest0-21swapper/322:10:103
1937399179,6cyclictest0-21swapper/300:20:343
1937399172,9cyclictest0-21swapper/319:20:123
1937399172,9cyclictest0-21swapper/319:20:113
1937399172,10cyclictest0-21swapper/321:05:143
1937399172,10cyclictest0-21swapper/321:05:133
1937399172,10cyclictest0-21swapper/320:10:133
1937399172,10cyclictest0-21swapper/320:10:123
1937399172,10cyclictest0-21swapper/320:05:133
1937399172,10cyclictest0-21swapper/320:00:123
1937399172,10cyclictest0-21swapper/319:15:123
1937399171,13cyclictest0-21swapper/319:10:153
1937399171,12cyclictest0-21swapper/323:25:133
1936199171,7cyclictest19324-21cyclictest19:10:022
1934899176,8cyclictest19364-21latency_hist19:10:011
1933699173,3cyclictest131rcu_preempt19:25:010
1933699173,3cyclictest131rcu_preempt19:25:010
1933699173,11cyclictest18488-21cut20:15:190
1933699173,10cyclictest0-21swapper/019:30:110
19336991712,3cyclictest0-21swapper/020:50:150
19336991712,3cyclictest0-21swapper/020:20:200
19336991711,4cyclictest0-21swapper/020:25:140
1933699171,13cyclictest26058-21kthreadcore23:25:190
1937399162,10cyclictest0-21swapper/319:25:133
1933699152,5cyclictest28031-21cpuspeed21:00:130
1937399141,8cyclictest0-21swapper/320:30:023
1933699141,9cyclictest19395-21latency_hist19:10:020
1937399133,8cyclictest0-21swapper/320:30:273
1937399131,10cyclictest0-21swapper/320:55:173
1937399124,5cyclictest0-21swapper/320:35:133
1937399122,7cyclictest0-21swapper/319:45:123
1937399122,6cyclictest0-21swapper/321:00:133
1937399101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional