You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 23:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Jul 08, 2026 12:45:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24501995427,19cyclictest0-21swapper/311:35:103
24501994924,16cyclictest35966irq/58-eth010:50:123
2449099491,38cyclictest26802-21ssh09:55:132
2447899450,40cyclictest17837-21apt-get09:50:121
2450199444,24cyclictest35966irq/58-eth011:10:123
2450199434,26cyclictest35966irq/58-eth010:20:123
2447899430,39cyclictest5294-21apt-get10:55:141
2447899430,39cyclictest5294-21apt-get10:55:131
2447899430,38cyclictest28609-21apt-get09:20:111
2450199421,27cyclictest35966irq/58-eth011:55:143
2447899422,37cyclictest24982-21apt-get10:30:131
2447899422,37cyclictest24982-21apt-get10:30:121
2447899421,37cyclictest0-21swapper/108:10:131
2449099411,32cyclictest0-21swapper/209:20:122
2447899411,35cyclictest0-21swapper/107:40:141
2447899410,37cyclictest0-21swapper/107:55:131
2447899401,35cyclictest0-21swapper/109:15:161
2447899400,35cyclictest5607-21apt-get09:05:121
2450199393,28cyclictest35966irq/58-eth010:45:113
2450199392,24cyclictest35966irq/58-eth008:50:133
24490993916,20cyclictest0-21swapper/210:30:132
24490993916,20cyclictest0-21swapper/210:30:122
2447899392,32cyclictest32187-21apt-get11:10:121
2447899391,34cyclictest10504-21apt-get09:10:091
2447899391,22cyclictest29233-21ntp_states07:45:251
2450199384,25cyclictest35966irq/58-eth009:55:133
2447899381,34cyclictest0-21swapper/107:35:131
2447899381,33cyclictest0-21swapper/111:40:131
2447899381,32cyclictest31839-21apt-get08:25:121
2447899381,22cyclictest19549-21sh10:45:071
2447899381,22cyclictest16191-21sh11:18:351
2447899380,27cyclictest22189-21sh11:57:161
2450199374,22cyclictest35966irq/58-eth009:35:143
24501993719,12cyclictest35966irq/58-eth012:25:123
24501993715,15cyclictest35966irq/58-eth009:25:143
2447899371,32cyclictest6883-21ssh11:50:121
2447899371,32cyclictest5223-21apt-get09:25:151
2447899371,32cyclictest28685-21apt-get10:50:121
2447899371,32cyclictest12375-21apt-get10:05:131
2447899371,30cyclictest17785-21apt-get11:20:121
2447899370,25cyclictest8746-21sshd08:34:141
2447899370,25cyclictest4208-21ntp_192.168.11409:00:241
24490993614,14cyclictest0-21swapper/208:10:122
2447899361,32cyclictest1557-21apt-get10:35:121
2447899361,32cyclictest0-21swapper/109:35:131
2447899360,33cyclictest30381-21apt-get10:15:121
2447899360,23cyclictest29180-21sshd07:13:121
24467993612,7cyclictest10157-21diskmemload10:01:310
2450199357,20cyclictest35966irq/58-eth011:30:133
2450199354,19cyclictest35966irq/58-eth010:25:133
2450199354,19cyclictest35966irq/58-eth010:25:123
24501993527,6cyclictest35966irq/58-eth012:00:133
2450199352,21cyclictest35966irq/58-eth008:45:133
2447899353,29cyclictest22876-21apt-get12:35:111
2447899351,31cyclictest3434-21apt-get11:30:131
2447899351,31cyclictest0-21swapper/112:00:131
2447899351,31cyclictest0-21swapper/111:00:131
2447899351,31cyclictest0-21swapper/111:00:131
2447899350,31cyclictest0-21swapper/108:20:121
2447899350,31cyclictest0-21swapper/108:20:121
2447899350,23cyclictest29409-21sshd07:15:121
2447899350,23cyclictest29409-21sshd07:15:111
2447899350,22cyclictest18791-21apt-get08:45:131
2447899350,21cyclictest23210-21apt-get11:05:131
24490993424,7cyclictest16163-21apt-get07:35:112
24490993422,8cyclictest9069-21apt-get08:35:142
24490993422,8cyclictest9069-21apt-get08:35:132
2449099341,15cyclictest19622-21ssh10:45:102
2447899341,23cyclictest86666ssh12:20:321
2447899340,31cyclictest5253-21ntp_states07:20:251
2447899340,25cyclictest20241-21sh12:32:171
2447899340,20cyclictest11608-21ssh12:27:391
24467993424,7cyclictest8852-21ssh09:45:130
24467993419,12cyclictest28390-21apt-get08:55:120
24490993328,2cyclictest30565-21apt-get07:50:132
24490993324,7cyclictest26886-21apt-get11:25:132
24490993323,7cyclictest1420-21apt-get12:05:142
24490993323,7cyclictest1420-21apt-get12:05:132
24490993320,7cyclictest27-21ksoftirqd/209:00:142
24490993314,15cyclictest0-21swapper/209:10:112
24490993314,14cyclictest0-21swapper/208:50:122
24490993311,11cyclictest0-21swapper/211:20:122
2447899331,29cyclictest8841-21apt-get09:45:121
2447899331,29cyclictest26790-21apt-get09:55:131
2447899330,2cyclictest0-21swapper/108:50:131
2447899330,2cyclictest0-21swapper/108:15:121
24467993322,8cyclictest5250-21ssh09:25:150
2450199329,18cyclictest33-21ksoftirqd/310:35:123
2449099328,16cyclictest10157-21diskmemload10:58:062
2449099328,16cyclictest10157-21diskmemload10:58:052
2449099325,18cyclictest24877-21apt-get12:00:122
24490993221,4cyclictest13930-21apt-get12:30:122
24490993218,11cyclictest1581-21ssh10:35:112
2449099321,10cyclictest0-21swapper/211:50:112
2447899321,28cyclictest9285-21cut08:00:191
2447899321,27cyclictest9834-21wc07:25:231
24478993210,19cyclictest18282-21usage_EthcTask08:40:301
2447899320,29cyclictest12364-21apt-get11:35:091
2447899320,28cyclictest12725-21df_inode08:05:131
2447899320,21cyclictest1-21systemd12:14:141
24467993225,5cyclictest10522-21ssh09:10:100
24467993222,8cyclictest17446-21apt-get08:10:140
24467993215,6cyclictest10157-21diskmemload10:53:250
24467993213,7cyclictest8711-21ssh11:15:090
24467993213,6cyclictest10157-21diskmemload12:05:030
24467993213,6cyclictest10157-21diskmemload12:05:020
24467993212,8cyclictest10157-21diskmemload12:10:470
24467993211,7cyclictest91-21pr/ttyS008:40:010
24467993211,7cyclictest91-21pr/ttyS008:40:000
2449099317,9cyclictest26868-21ssh12:18:382
2449099312,14cyclictest12583-21ssh09:45:282
24490993120,8cyclictest11355-21apt-get07:30:132
2447899311,27cyclictest7381-21wc10:00:301
2447899311,27cyclictest5713-21usage_Anybus12:05:291
2447899311,27cyclictest5713-21usage_Anybus12:05:281
2447899311,27cyclictest15631-21awk10:25:001
2447899311,26cyclictest30836-21cut11:45:181
2447899311,26cyclictest20054-21cat10:25:301
2447899311,26cyclictest20054-21cat10:25:291
2447899311,26cyclictest13836-21wc08:40:021
2447899311,26cyclictest13836-21wc08:40:021
24467993116,11cyclictest0-21swapper/010:30:130
24467993116,11cyclictest0-21swapper/010:30:120
24467993113,7cyclictest7516-21ssh09:43:520
24467993113,6cyclictest29642-21ssh12:37:480
24467993112,8cyclictest19496-21ssh11:38:150
24467993112,7cyclictest18777-21ssh10:07:240
24467993112,6cyclictest8709-21ssh11:31:010
24467993112,4cyclictest10157-21diskmemload11:02:090
24467993112,4cyclictest10157-21diskmemload11:02:080
2446799310,9cyclictest2728-21ssh12:22:460
24501993017,9cyclictest0-21swapper/312:15:133
24501993014,12cyclictest35966irq/58-eth010:10:153
2449099309,18cyclictest0-21swapper/207:55:142
24490993015,12cyclictest11328-21cut11:15:212
24490993012,16cyclictest0-21swapper/209:50:112
24490993012,15cyclictest0-21swapper/209:25:142
2447899302,23cyclictest26898-21ssh11:25:121
2447899301,26cyclictest32695-21irqcore09:40:191
2447899301,26cyclictest31119-21cut07:50:181
2447899301,25cyclictest11565-21sed07:30:171
2447899300,27cyclictest21324-21/usr/sbin/munin10:10:131
2447899300,27cyclictest0-21swapper/112:15:121
24467993018,8cyclictest21934-21grep10:10:190
24467993013,6cyclictest12030-21ssh11:50:500
24467993012,8cyclictest13620-21ssh10:59:490
24467993012,8cyclictest13620-21ssh10:59:490
24467993012,7cyclictest8371-21ssh11:15:000
24467993012,7cyclictest4752-21ssh09:24:560
24467993012,7cyclictest26789-21ssh09:18:190
24467993012,7cyclictest25984-21ssh11:24:290
24467993012,7cyclictest21137-21ssh11:55:590
24467993011,7cyclictest28166-21ssh11:05:370
24467993011,7cyclictest1489-21ssh11:28:060
24467993010,8cyclictest10414-21ssh12:26:120
24467993010,11cyclictest10157-21diskmemload11:40:440
2446799300,13cyclictest14796-21awk10:40:300
2449099295,8cyclictest32114-21ssh11:10:072
2449099292,18cyclictest10157-21diskmemload11:37:282
24490992920,7cyclictest10591-21apt-get10:40:132
24490992918,8cyclictest0-21swapper/208:55:132
24490992913,13cyclictest2980-21sendmail_mailtr08:25:252
2449099291,23cyclictest0-21swapper/211:59:112
2447899291,25cyclictest29029-21/usr/sbin/munin08:55:191
2447899291,25cyclictest11103-21cut10:40:171
2447899291,23cyclictest14219-21apt-get09:30:131
24467992923,3cyclictest30329-21ssh11:45:120
24467992919,7cyclictest17864-21ssh09:50:120
24467992915,10cyclictest7852-21apt-get08:00:100
24467992911,7cyclictest21098-21ssh10:25:480
24467992911,7cyclictest21098-21ssh10:25:470
24467992911,6cyclictest7549-21ssh10:36:510
24467992911,6cyclictest12152-21ssh10:20:510
2450199282,14cyclictest35966irq/58-eth007:25:133
2449099283,16cyclictest0-21swapper/210:25:152
2449099283,16cyclictest0-21swapper/210:25:152
24490992819,6cyclictest30379-21ssh10:15:112
24490992818,7cyclictest1786-21apt-get07:20:132
24490992816,9cyclictest0-21swapper/210:05:142
24490992814,11cyclictest0-21swapper/209:15:152
24490992813,12cyclictest11570-21cat08:00:262
2449099281,21cyclictest5112-21diskstats12:25:152
24490992812,13cyclictest10135-21awk07:25:272
24490992810,9cyclictest5489-21ssh11:48:472
2447899280,25cyclictest24487-21cat07:10:011
24467992812,6cyclictest31272-21ssh12:02:230
24467992812,6cyclictest24915-21ssh12:16:160
24467992811,6cyclictest31651-21ssh09:39:460
24467992810,7cyclictest2348-21ssh09:59:170
2450199273,17cyclictest0-21swapper/311:54:093
2450199273,16cyclictest0-21swapper/312:20:143
2450199273,15cyclictest0-21swapper/312:12:143
2450199273,12cyclictest35966irq/58-eth011:40:133
2449099276,9cyclictest930-21snmpd10:23:482
24490992719,5cyclictest27346irq/40-dwc2_hso07:45:152
24490992712,8cyclictest3821-21ssh10:53:452
24490992712,12cyclictest27143-21cat11:05:292
24490992712,12cyclictest0-21swapper/209:30:132
24490992711,8cyclictest26913-21/usr/sbin/munin09:35:262
24490992711,14cyclictest29369-21sshd10:14:142
24490992711,12cyclictest8844-21ssh10:01:142
24490992710,14cyclictest26279-21ssh11:40:342
2449099270,12cyclictest3990-21ssh12:24:172
24467992714,8cyclictest14222-21ssh09:30:130
24467992711,6cyclictest30356-21ssh10:15:100
2446799270,9cyclictest24393-21usage_EthcTask10:45:290
2446799270,9cyclictest14023-21entropy08:40:150
2446799270,12cyclictest15125-21timerwakeupswit07:30:280
2450199266,11cyclictest35966irq/58-eth009:40:123
2450199264,12cyclictest35966irq/58-eth011:00:123
2450199264,12cyclictest35966irq/58-eth011:00:113
2450199263,15cyclictest35966irq/58-eth012:35:123
2450199263,15cyclictest35966irq/58-eth009:10:103
24501992614,9cyclictest35966irq/58-eth010:00:133
24490992615,8cyclictest22879-21ssh12:35:102
24490992613,11cyclictest10567-21fschecks_count12:10:152
24490992613,10cyclictest12969-21ps08:05:172
24490992611,12cyclictest30462-21grep08:20:242
24490992611,12cyclictest30462-21grep08:20:232
24490992610,12cyclictest32280-21apt09:40:132
24467992617,7cyclictest6287-21kthreadcore08:30:180
24467992617,7cyclictest10157-21diskmemload12:34:290
24467992617,6cyclictest24239-21irqstats08:50:190
24467992616,8cyclictest25436-21kthreadcore07:10:180
24467992616,7cyclictest6584-21apt-get07:25:150
24467992611,10cyclictest0-21swapper/007:35:130
2450199255,11cyclictest0-21swapper/311:15:133
2450199251,17cyclictest35966irq/58-eth008:10:133
24501992510,7cyclictest41-21kworker/3:110:05:123
2449099259,8cyclictest7662-21awk08:30:242
2449099259,13cyclictest12134-21sed11:35:012
2449099251,10cyclictest0-21swapper/208:15:252
2446799257,11cyclictest12-21ksoftirqd/009:05:120
24467992514,8cyclictest73-21kswapd008:45:300
2450199243,13cyclictest0-21swapper/309:30:143
2449099249,12cyclictest5886-21if_eth009:05:172
2449099248,13cyclictest18478-21latency_hist08:45:012
24490992418,3cyclictest14200-21ssh11:00:132
24490992418,3cyclictest14200-21ssh11:00:122
24490992410,7cyclictest28448-21unixbench_multi07:10:282
24467992416,6cyclictest20969-21apt-get07:40:120
2446799241,18cyclictest785-21apt-get09:00:140
24467992411,10cyclictest12636-21apt-get08:05:120
2446799240,9cyclictest25877-21/usr/sbin/munin07:45:160
2450199231,13cyclictest35966irq/58-eth011:45:133
2450199231,12cyclictest35966irq/58-eth007:30:123
2449099238,12cyclictest0-21swapper/207:40:132
24490992311,8cyclictest1665-21cut07:20:022
24490992311,8cyclictest1665-21cut07:20:012
24490992310,10cyclictest23364-21cron08:50:012
24467992315,6cyclictest3036-21apt-get07:55:140
24467992314,7cyclictest5381-21mailstats07:20:270
24467992313,8cyclictest2916-21latency_hist07:55:010
2450199224,11cyclictest0-21swapper/309:15:143
24467992213,6cyclictest0-21swapper/008:15:120
2450199213,13cyclictest0-21swapper/310:15:123
24501992113,4cyclictest0-21swapper/311:25:133
2449099216,12cyclictest24519-21latency_hist07:10:022
24467992114,5cyclictest30449-21ntp_192.168.11408:20:240
24467992114,5cyclictest30449-21ntp_192.168.11408:20:230
24467992112,7cyclictest31934-21sshd08:25:140
24501992011,6cyclictest0-21swapper/312:05:213
24501992011,6cyclictest0-21swapper/312:05:203
2450199201,12cyclictest0-21swapper/309:00:173
24501992010,8cyclictest0-21swapper/310:32:393
24501992010,8cyclictest0-21swapper/310:32:393
24501992010,7cyclictest35966irq/58-eth007:55:143
2450199199,8cyclictest0-21swapper/312:30:053
2450199198,8cyclictest0-21swapper/310:57:143
2450199198,8cyclictest0-21swapper/310:57:133
2450199197,8cyclictest35966irq/58-eth009:20:123
24501991911,6cyclictest35966irq/58-eth011:20:133
2450199191,13cyclictest0-21swapper/309:05:283
2450199191,13cyclictest0-21swapper/307:35:163
2450199191,12cyclictest0-21swapper/307:45:003
2450199191,12cyclictest0-21swapper/307:15:243
2450199191,12cyclictest0-21swapper/307:15:233
24501991910,6cyclictest0-21swapper/310:43:253
2450199182,10cyclictest0-21swapper/308:40:133
2450199182,10cyclictest0-21swapper/307:20:133
24501991812,3cyclictest33-21ksoftirqd/308:30:143
24501991810,6cyclictest0-21swapper/309:46:143
24467991812,4cyclictest29587-21fschecks_time07:15:170
24467991812,4cyclictest29587-21fschecks_time07:15:170
2450199172,10cyclictest0-21swapper/309:50:133
2450199172,10cyclictest0-21swapper/308:25:133
2450199172,10cyclictest0-21swapper/307:45:153
24501991711,4cyclictest0-21swapper/311:05:263
2450199161,12cyclictest0-21swapper/308:05:023
2450199161,10cyclictest0-21swapper/308:05:153
2450199151,9cyclictest0-21swapper/308:35:153
2450199151,9cyclictest0-21swapper/308:35:143
2450199151,7cyclictest0-21swapper/308:20:123
2450199151,7cyclictest0-21swapper/308:20:113
2450199151,11cyclictest0-21swapper/307:10:123
24501991510,3cyclictest33-21ksoftirqd/308:55:293
24501991510,3cyclictest33-21ksoftirqd/307:50:283
2446799140,10cyclictest24542-21latency_hist07:10:020
2450199121,5cyclictest0-21swapper/308:15:193
2450199103,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional