You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 08:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Dec 19, 2025 00:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1902722660,5sleep10-21swapper/119:10:041
1777321390,4sleep20-21swapper/219:05:242
1964521300,8sleep01921399cyclictest22:10:190
2372321000,2sleep121-21rcuc/120:50:301
160242960,3sleep10-21swapper/122:00:181
1921399891,39cyclictest0-21swapper/023:50:160
1921399891,39cyclictest0-21swapper/023:50:160
258892880,2sleep20-21swapper/222:25:302
32032870,2sleep00-21swapper/019:55:190
273912740,2sleep10-21swapper/100:00:241
19216997227,21cyclictest13934-21awk21:55:053
19216997227,21cyclictest13934-21awk21:55:053
19213996715,11cyclictest0-21swapper/019:50:180
298902660,4sleep3381ktimersoftd/319:40:053
19216996624,12cyclictest26897-21latency_hist00:00:043
1921399663,44cyclictest7004-21aten_rbpower_en21:35:200
1921399648,52cyclictest32455-21missed_timers19:45:250
1921399646,42cyclictest1708-21modprobe00:20:040
19216996326,7cyclictest5289-21apt-get23:00:173
19216996326,19cyclictest16368-21if_eth022:00:223
19216996322,13cyclictest28157-21cat19:35:053
1921399637,35cyclictest8237-21grep23:05:350
1921399636,44cyclictest29866-21cat19:40:050
1921399635,54cyclictest101ktimersoftd/022:45:340
1921399634,45cyclictest26786-21cron00:00:040
19216996230,17cyclictest27704-21/usr/sbin/munin00:00:303
19216996230,14cyclictest9795-21ls21:40:333
1921399626,43cyclictest23317-21fschecks_time20:50:240
1921399625,35cyclictest9880-21head21:40:340
19216996132,15cyclictest6944-21apt-get21:35:173
19216996129,20cyclictest20284-21awk23:40:243
19216996124,12cyclictest23230-21df20:50:223
1921399611,47cyclictest21331-21wget20:45:180
19213996114,15cyclictest0-21swapper/022:25:160
19216996027,7cyclictest394-21ls19:45:313
19216996024,10cyclictest7950irq/56-dwc_otg_20:40:043
19216996023,10cyclictest7950irq/56-dwc_otg_23:00:043
1921399607,26cyclictest0-21swapper/020:30:180
19213996034,10cyclictest0-21swapper/021:15:230
1921399601,54cyclictest26675-21basename21:00:180
19213996013,5cyclictest101ktimersoftd/021:10:040
19213996010,34cyclictest24346-21unixbench_singl22:20:330
19216995929,9cyclictest0-21swapper/320:15:173
19216995926,12cyclictest2182-21tune2fs00:20:223
1921599591,17cyclictest7850irq/56-dwc_otg_20:00:182
1921399599,38cyclictest0-21swapper/022:40:110
1921399595,37cyclictest16648-21ntp_kernel_pll_22:00:270
1921399591,31cyclictest27040-21runrttasks19:30:240
19213995815,39cyclictest101ktimersoftd/020:20:340
19216995732,12cyclictest32312-21/usr/sbin/munin22:45:363
19216995718,21cyclictest12605-21/usr/sbin/munin23:20:213
19216995624,12cyclictest7850irq/56-dwc_otg_21:15:303
1921399566,32cyclictest0-21swapper/000:20:260
19213995638,5cyclictest101ktimersoftd/023:30:290
19216995526,8cyclictest8270-21users23:05:353
19216995524,13cyclictest1124-21snmpd00:16:143
19216995523,10cyclictest21120-21latency_hist20:45:053
19213995540,5cyclictest101ktimersoftd/019:40:270
19216995421,13cyclictest0-21swapper/300:05:263
1921399548,29cyclictest11509-21grep20:15:340
1921399542,40cyclictest0-21swapper/023:45:250
19213995420,4cyclictest0-21swapper/020:55:180
19213995414,22cyclictest0-21swapper/023:15:170
19216995323,16cyclictest21423-21/usr/sbin/munin20:45:193
19216995322,11cyclictest17754-21/usr/sbin/munin22:05:283
19216995320,11cyclictest7950irq/56-dwc_otg_20:12:363
19213995341,3cyclictest0-21swapper/021:49:050
19213995325,10cyclictest101ktimersoftd/021:55:050
19213995325,10cyclictest101ktimersoftd/021:55:050
19213995320,10cyclictest17754-21/usr/sbin/munin22:05:270
1921399531,31cyclictest4483-21unixbench_singl21:25:330
19216995225,12cyclictest23932-21latency23:50:253
19216995225,12cyclictest23932-21latency23:50:253
19216995218,11cyclictest0-21swapper/323:30:253
1921399525,36cyclictest0-21swapper/023:11:180
1921399524,44cyclictest19890-21/usr/sbin/munin23:40:190
19213995236,7cyclictest0-21swapper/019:19:100
19213995234,8cyclictest0-21swapper/020:26:010
19213995234,8cyclictest0-21swapper/020:26:010
1921399521,31cyclictest0-21swapper/000:00:310
19216995127,11cyclictest24249-21chrt22:20:323
19216995121,11cyclictest128-21systemd-journal23:37:103
19215995123,5cyclictest1188-21idleruntime-cro21:20:042
1921399517,25cyclictest0-21swapper/000:10:200
19213995131,8cyclictest0-21swapper/022:34:170
19213995130,9cyclictest0-21swapper/023:00:160
19213995128,4cyclictest128-21systemd-journal20:00:250
19213995121,9cyclictest3527-21aten_rbpower_cu22:55:190
1921699508,9cyclictest7950irq/56-dwc_otg_22:15:153
19216995022,11cyclictest30633-21ls19:40:243
19216995020,15cyclictest7950irq/56-dwc_otg_21:32:113
19216995020,11cyclictest22537-21seq19:19:093
19213995033,8cyclictest0-21swapper/022:51:240
19216994927,12cyclictest4933-21chrt00:25:423
19216994927,12cyclictest4933-21chrt00:25:423
19216994920,10cyclictest7750irq/56-dwc_otg22:40:223
19216994917,9cyclictest7950irq/56-dwc_otg_20:34:163
1921399498,5cyclictest101ktimersoftd/023:25:330
19213994922,6cyclictest0-21swapper/021:10:230
19213994921,5cyclictest22972-21idleruntime-cro22:20:040
1921399491,4cyclictest19774-21ls20:40:230
1921399491,36cyclictest8326-21munin-run20:10:040
19216994838,7cyclictest26843-21df21:00:203
19216994821,12cyclictest11824-21runrttasks21:46:363
19216994821,11cyclictest15656-21sshd23:26:103
19216994817,19cyclictest7950irq/56-dwc_otg_21:13:033
1921499482,12cyclictest7750irq/56-dwc_otg20:20:171
19213994825,6cyclictest7850irq/56-dwc_otg_21:20:220
19216994732,11cyclictest28861-21/usr/sbin/munin21:05:243
19216994724,13cyclictest26169-21taskset19:29:213
19216994722,11cyclictest128-21systemd-journal21:25:423
19216994718,14cyclictest0-21swapper/319:24:243
1921599477,25cyclictest0-21swapper/223:25:182
1921399474,11cyclictest0-21swapper/000:30:170
19213994728,9cyclictest8643-21taskset00:37:020
19213994717,5cyclictest0-21swapper/022:35:220
19216994620,11cyclictest7006-21taskset00:34:113
19216994618,11cyclictest19567-21/usr/sbin/munin22:10:183
1921499464,12cyclictest7850irq/56-dwc_otg_22:45:161
19214994620,11cyclictest7950irq/56-dwc_otg_19:15:171
19213994630,3cyclictest0-21swapper/021:32:070
19213994625,6cyclictest101ktimersoftd/022:00:040
19213994619,7cyclictest19670-21latency_hist23:40:050
19213994619,14cyclictest0-21swapper/000:09:060
1921399461,6cyclictest12855-21fschecks_count23:20:230
1921399460,43cyclictest19176-21latency_hist20:40:040
19216994530,11cyclictest7591-21/usr/sbin/munin20:05:303
19216994528,14cyclictest12540-21perl20:20:233
1921499454,30cyclictest0-21swapper/119:50:181
1921499452,8cyclictest0-21swapper/122:40:181
19213994517,7cyclictest8625-21aten_rbpower_en20:10:190
1921399451,41cyclictest0-21swapper/019:20:200
19216994435,7cyclictest7334-21apt-get00:35:183
19216994434,8cyclictest21632-21apt-get23:45:153
19216994430,11cyclictest12514-21latency_hist23:20:053
19216994429,12cyclictest31164-21if_eth000:10:243
19216994429,12cyclictest26601-21sed22:30:053
19216994428,12cyclictest5169-21diskstats20:00:203
19216994420,11cyclictest7950irq/56-dwc_otg_21:21:043
19216994418,12cyclictest0-21swapper/320:30:053
19216994418,12cyclictest0-21swapper/320:30:053
1921599447,25cyclictest0-21swapper/222:35:192
1921599447,24cyclictest0-21swapper/200:05:092
1921599447,23cyclictest0-21swapper/222:00:172
1921599447,22cyclictest0-21swapper/222:55:192
1921599445,26cyclictest0-21swapper/219:10:182
1921599444,5cyclictest7950irq/56-dwc_otg_22:50:172
1921499443,26cyclictest0-21swapper/122:25:161
1921499442,8cyclictest0-21swapper/119:20:141
1921399443,32cyclictest4935-21taskset00:25:430
1921399443,32cyclictest4935-21taskset00:25:430
117082440,4sleep2301ktimersoftd/221:45:332
19216994333,8cyclictest24834-21apt-get20:55:163
1921599437,26cyclictest0-21swapper/221:20:092
1921599433,9cyclictest0-21swapper/223:55:152
19215994319,4cyclictest24251-21sensors23:50:302
19215994319,4cyclictest24251-21sensors23:50:302
1921499436,12cyclictest7850irq/56-dwc_otg_23:35:161
19216994228,11cyclictest26798-21/usr/sbin/munin22:30:333
19216994228,11cyclictest19943-21ntp_states19:10:283
19216994227,12cyclictest9886-21sed23:10:323
19216994227,12cyclictest28593-21sshd22:35:103
19216994226,12cyclictest2520-21taskset19:51:393
19216994219,12cyclictest1673-21/usr/sbin/munin22:50:193
19216994219,11cyclictest4417-21runrttasks19:56:343
1921599422,4cyclictest0-21swapper/221:50:182
1921599422,4cyclictest0-21swapper/221:50:182
1921599421,14cyclictest0-21swapper/200:15:322
19214994219,4cyclictest27668-21vmstat19:30:321
1921399421,25cyclictest19232-21/usr/sbin/munin19:10:230
19216994127,11cyclictest14855-21ntp_states21:55:273
1921599417,25cyclictest0-21swapper/219:40:172
1921499416,9cyclictest7750irq/56-dwc_otg00:30:051
1921499416,9cyclictest7750irq/56-dwc_otg00:30:051
19214994111,11cyclictest0-21swapper/121:00:271
1873524123,12sleep00-21swapper/019:07:290
19215994011,16cyclictest26594-21wget19:30:172
1921499403,6cyclictest7750irq/56-dwc_otg20:55:181
1921499402,27cyclictest7948-21ls23:05:321
19214994017,4cyclictest7850irq/56-dwc_otg_23:30:051
1921499401,11cyclictest14091-21wget20:25:181
1921499401,11cyclictest14091-21wget20:25:181
271992390,5sleep21921599cyclictest21:00:252
1921399391,21cyclictest466-21in:imuxsock19:25:190
106042390,17sleep17950irq/56-dwc_otg_23:15:041
55332370,4sleep25537-21acpi00:30:162
19215993711,13cyclictest0-21swapper/223:45:172
1921499379,9cyclictest0-21swapper/123:50:161
1921499379,9cyclictest0-21swapper/123:50:161
1921499375,7cyclictest7850irq/56-dwc_otg_20:45:221
1921599366,15cyclictest7850irq/56-dwc_otg_20:40:232
1921599361,24cyclictest4088-21ls00:25:222
1921599361,24cyclictest4088-21ls00:25:222
1921499361,24cyclictest30613-21ls21:10:241
1921499361,22cyclictest32121-21cut19:45:211
1921599355,6cyclictest7850irq/56-dwc_otg_19:50:352
19215993513,10cyclictest301ktimersoftd/223:33:442
1921499354,15cyclictest7950irq/56-dwc_otg_23:20:191
1921499354,15cyclictest7950irq/56-dwc_otg_19:45:051
1921499351,5cyclictest7750irq/56-dwc_otg20:05:181
1921499351,25cyclictest23-21ksoftirqd/122:55:311
1921599348,11cyclictest7950irq/56-dwc_otg_21:00:042
1921599346,6cyclictest7850irq/56-dwc_otg_23:05:192
1921599344,5cyclictest7850irq/56-dwc_otg_23:10:262
1921599344,10cyclictest7850irq/56-dwc_otg_19:35:322
1921599343,17cyclictest7950irq/56-dwc_otg_22:35:052
1921599341,3cyclictest7750irq/56-dwc_otg20:25:152
1921599341,3cyclictest7750irq/56-dwc_otg20:25:152
1921499347,6cyclictest7750irq/56-dwc_otg20:15:341
1921499347,5cyclictest7850irq/56-dwc_otg_23:00:191
1921499347,12cyclictest7950irq/56-dwc_otg_22:40:051
1921499343,15cyclictest7850irq/56-dwc_otg_22:10:251
1921499343,12cyclictest7950irq/56-dwc_otg_19:25:321
19214993428,4cyclictest23-21ksoftirqd/121:40:341
1921499341,4cyclictest7750irq/56-dwc_otg22:15:321
1921499341,24cyclictest30658-21ls00:10:051
1921599337,14cyclictest7950irq/56-dwc_otg_22:10:182
1921599334,4cyclictest11963-21grep23:15:332
1921599332,6cyclictest24998-21cut19:25:212
1921599331,25cyclictest1100-21ls19:50:052
1921499338,5cyclictest7750irq/56-dwc_otg19:10:161
1921499338,22cyclictest0-21swapper/122:30:191
1921499333,5cyclictest7750irq/56-dwc_otg20:35:301
1921499332,21cyclictest11564-21timerwakeupswit21:45:321
1921499332,19cyclictest0-21swapper/123:43:491
1921499332,10cyclictest22724-21grep23:45:321
1921499331,25cyclictest26181-21timerandwakeup23:55:301
1921499331,25cyclictest1948-21cut22:50:231
1921499331,24cyclictest2873-21timerandwakeup00:20:321
1921499331,21cyclictest23524-21cut22:20:221
19214993311,5cyclictest3115-21gdbus19:55:061
1921599327,15cyclictest0-21swapper/222:40:102
1921599324,6cyclictest21668-21fschecks_time22:15:212
1921599324,14cyclictest7950irq/56-dwc_otg_23:00:332
1921599321,8cyclictest12231-21grep20:20:192
1921599321,19cyclictest3904-21missed_timers21:25:262
1921599321,18cyclictest435-21cron20:00:042
1921599321,11cyclictest0-21swapper/200:35:242
1921499327,22cyclictest0-21swapper/121:55:171
1921499327,22cyclictest0-21swapper/121:25:191
1921499325,4cyclictest7057-21grep21:35:201
1921499325,12cyclictest7950irq/56-dwc_otg_00:15:201
19214993213,17cyclictest0-21swapper/100:10:201
19214993211,7cyclictest0-21swapper/120:30:231
1921599317,15cyclictest0-21swapper/200:00:202
1921599315,5cyclictest7850irq/56-dwc_otg_19:15:232
1921599314,5cyclictest7750irq/56-dwc_otg22:45:192
1921599311,6cyclictest7950irq/56-dwc_otg_22:05:332
1921599311,24cyclictest29357-21timerandwakeup21:05:322
1921599311,16cyclictest0-21swapper/221:31:012
1921499317,22cyclictest0-21swapper/121:30:181
1921499317,10cyclictest7950irq/56-dwc_otg_20:00:241
19214993113,4cyclictest20115-21ntpq20:40:281
1921499311,18cyclictest0-21swapper/100:35:101
1921499311,12cyclictest7750irq/56-dwc_otg21:20:341
19214993110,19cyclictest0-21swapper/123:15:181
1921599307,5cyclictest11036-21/usr/sbin/munin20:15:272
1921599307,14cyclictest0-21swapper/220:10:182
1921599307,14cyclictest0-21swapper/200:20:192
1921599307,10cyclictest7950irq/56-dwc_otg_20:30:262
1921599306,5cyclictest7950irq/56-dwc_otg_21:35:342
19215993022,5cyclictest1141-21lldpd22:20:282
1921599302,19cyclictest0-21swapper/223:23:012
1921599302,17cyclictest7950irq/56-dwc_otg_20:35:312
1921599302,16cyclictest0-21swapper/219:20:272
1921599302,14cyclictest7950irq/56-dwc_otg_23:40:042
19215993020,7cyclictest31077-21fschecks_time00:10:242
1921599301,4cyclictest7750irq/56-dwc_otg23:40:152
1921499306,22cyclictest0-21swapper/121:05:191
1867123018,6sleep30-21swapper/319:06:393
1921599295,6cyclictest10333-21modprobe21:45:042
1921599292,6cyclictest7850irq/56-dwc_otg_21:10:282
1921599292,18cyclictest0-21swapper/220:05:292
1921599291,17cyclictest21455-21uniq20:45:212
1921499294,4cyclictest7950irq/56-dwc_otg_20:10:301
1921499291,4cyclictest0-21swapper/122:05:181
1921599281,3cyclictest7850irq/56-dwc_otg_20:50:272
1921499286,5cyclictest7850irq/56-dwc_otg_19:35:151
1921499285,4cyclictest7950irq/56-dwc_otg_21:15:151
1921499281,4cyclictest7950irq/56-dwc_otg_23:35:041
1921599261,15cyclictest0-21swapper/221:55:212
1921499267,6cyclictest0-21swapper/100:31:101
1921499262,6cyclictest7950irq/56-dwc_otg_21:53:261
1921499262,6cyclictest7950irq/56-dwc_otg_21:53:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional