You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-16 - 14:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Dec 16, 2025 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1856921020,2sleep318570-21/usr/sbin/munin22:58:053
1693999725,60cyclictest26879-21sed23:18:180
1693999628,50cyclictest11256-21/usr/sbin/munin22:38:030
169399948,74cyclictest0-21swapper/023:53:020
1693999427,57cyclictest13534-21kernelversion22:43:080
1693999427,57cyclictest13534-21kernelversion22:43:080
1693999227,56cyclictest0-21swapper/021:09:380
1693999227,53cyclictest0-21swapper/019:59:270
1693999129,47cyclictest28812-21sh21:55:030
1693999129,47cyclictest28812-21sh21:55:030
1693999124,47cyclictest26487-21timerandwakeup20:18:170
1693998930,48cyclictest0-21swapper/019:38:000
1693998930,41cyclictest5031-21taskset19:19:290
1693998929,41cyclictest19072-21/usr/sbin/munin00:28:080
1693998729,55cyclictest16475-21wc00:20:040
1693998729,55cyclictest16475-21wc00:20:040
1693998723,56cyclictest0-21swapper/020:53:120
1693998631,41cyclictest23266-21grep23:08:170
1693998628,45cyclictest0-21swapper/000:23:070
1693998628,39cyclictest24398-21ntp_offset20:13:140
1693998625,49cyclictest31427-21idleruntime22:03:080
1693998524,49cyclictest18590-21/usr/sbin/munin21:28:070
1693998425,49cyclictest24877-21cat23:13:150
1693998424,45cyclictest21634-21chrt00:33:290
1693998421,49cyclictest0-21swapper/020:48:100
1693998327,43cyclictest29553-21fschecks_time21:58:060
1693998225,41cyclictest0-21swapper/021:01:350
1693998224,44cyclictest0-21swapper/022:54:060
1693998127,40cyclictest26956-21taskset21:49:360
1693998127,35cyclictest0-21swapper/023:39:470
1693998025,45cyclictest0-21swapper/019:47:590
1693998025,45cyclictest0-21swapper/019:47:590
1693998025,37cyclictest16154-21latency_hist21:20:030
1693998023,48cyclictest0-21swapper/022:30:150
1693997926,38cyclictest0-21swapper/021:33:530
1693997923,49cyclictest0-21swapper/023:48:150
1693997922,49cyclictest0-21swapper/023:29:190
1693997922,48cyclictest0-21swapper/020:58:350
169399783,59cyclictest0-21swapper/019:10:050
1693997828,39cyclictest9015-21chrt23:58:360
1693997826,44cyclictest0-21swapper/021:22:440
1693997823,44cyclictest0-21swapper/021:15:040
169399776,38cyclictest0-21swapper/021:35:080
1693997727,31cyclictest0-21swapper/023:20:590
1693997727,31cyclictest0-21swapper/023:20:590
1693997722,42cyclictest0-21swapper/019:54:050
1693997628,33cyclictest10848-21chrt00:03:590
1693997621,40cyclictest128-21systemd-journal22:13:520
1693997528,33cyclictest16091-21sshd22:48:440
1693997520,38cyclictest0-21swapper/019:27:150
61072740,2sleep20-21swapper/222:23:072
1693997426,28cyclictest21088-21ls20:03:150
1693997424,40cyclictest0-21swapper/022:16:320
1693997424,39cyclictest28547-21chrt20:23:390
1693997424,39cyclictest28547-21chrt20:23:390
1693997424,38cyclictest30824-21chrt20:31:440
1693997420,40cyclictest0-21swapper/020:40:060
1693997332,28cyclictest21599-21mkdir23:04:550
1693997325,32cyclictest5760-21taskset22:21:520
1693997323,40cyclictest0-21swapper/020:37:240
1693997319,39cyclictest22138-21fschecks_count20:08:070
1693997226,32cyclictest0-21swapper/000:09:210
1693997225,36cyclictest0-21swapper/000:15:030
1693997126,32cyclictest12605-21taskset19:40:400
1693997125,37cyclictest0-21swapper/022:08:310
1693997124,37cyclictest22092-21chrt00:36:150
1693997123,33cyclictest0-21swapper/021:41:340
1693997022,37cyclictest0-21swapper/023:31:440
1693997020,38cyclictest0-21swapper/022:57:120
1694996938,25cyclictest1698-21taskset19:10:051
1693996924,34cyclictest0-21swapper/019:30:150
1693996923,37cyclictest0-21swapper/020:26:210
1693996826,31cyclictest2157-21chrt23:40:080
1693996824,33cyclictest7616-21chrt22:27:350
1693996726,24cyclictest0-21swapper/019:22:150
1694996214,18cyclictest7750irq/56-dwc_otg00:18:041
1694996214,18cyclictest7750irq/56-dwc_otg00:18:041
1694996022,24cyclictest16896-21/usr/sbin/munin21:23:061
1694996022,22cyclictest0-21swapper/122:38:011
1694995821,12cyclictest6403-21ntp_states20:53:131
1694995628,8cyclictest9468-21apt-get22:33:011
1694995624,14cyclictest4644-21/usr/sbin/munin20:48:131
1694995620,12cyclictest18866-21cat21:28:101
13022563,10sleep00-21swapper/019:08:410
169699557,25cyclictest0-21swapper/300:18:043
169699557,25cyclictest0-21swapper/300:18:043
1694995521,15cyclictest19302-21ntpq00:28:111
1694995520,19cyclictest11219-21taskset19:38:061
1694995433,8cyclictest9203-21apt-get19:33:021
70572530,2sleep10-21swapper/119:25:041
1694995317,12cyclictest31280-21df_inode22:03:051
169699522,7cyclictest7950irq/56-dwc_otg_22:43:013
169699522,7cyclictest7950irq/56-dwc_otg_22:43:013
1694995219,12cyclictest23483-21taskset20:10:141
1694995217,19cyclictest26957-21taskset21:49:371
169699512,8cyclictest7950irq/56-dwc_otg_22:03:023
1696995118,13cyclictest1125-21runrttasks19:10:053
1694995123,11cyclictest4812-21sed23:48:101
1694995118,17cyclictest7950irq/56-dwc_otg_00:33:121
1696995015,12cyclictest17387-21ls00:23:103
169599507,6cyclictest0-21swapper/221:23:002
169499507,26cyclictest0-21swapper/120:18:041
1694995018,11cyclictest26686-21/usr/sbin/munin23:18:171
1694995014,11cyclictest17490-21netstat00:23:111
1694995012,14cyclictest29272-21runrttasks21:58:011
169699497,23cyclictest0-21swapper/320:00:083
1696994912,14cyclictest24431-21/usr/sbin/munin20:13:153
1694994913,14cyclictest30684-21taskset23:29:221
1694994823,10cyclictest30821-21sshd20:31:431
1694994821,12cyclictest25106-21sed23:13:171
1694994818,11cyclictest21355-21chrt21:33:311
1694994814,25cyclictest0-21swapper/119:18:021
169699477,23cyclictest0-21swapper/321:03:033
169499476,26cyclictest0-21swapper/123:05:081
1694994716,18cyclictest28996-21taskset20:26:261
1694994716,11cyclictest6793-21taskset23:53:131
169499467,25cyclictest0-21swapper/120:00:081
1694994618,14cyclictest18131-21chrt19:57:111
1694994616,11cyclictest1124-21snmpd20:58:351
1694994611,11cyclictest7850irq/56-dwc_otg_23:04:591
169699457,25cyclictest0-21swapper/323:05:083
169699456,23cyclictest7950irq/56-dwc_otg_22:38:063
1696994520,5cyclictest19546-21idleruntime-cro20:00:033
1696994515,7cyclictest23992-21apt-get23:13:013
169599457,25cyclictest0-21swapper/200:08:032
169599457,24cyclictest0-21swapper/220:28:022
1694994516,13cyclictest32691-21taskset22:05:461
169699447,22cyclictest0-21swapper/323:28:003
1696994412,9cyclictest6613-21ls20:53:163
169599447,25cyclictest0-21swapper/222:28:022
169499447,25cyclictest0-21swapper/121:03:031
169499447,23cyclictest0-21swapper/120:43:021
1694994419,13cyclictest9015-21apt22:30:031
1694994418,16cyclictest3916-21taskset22:16:351
1694994413,15cyclictest7750irq/56-dwc_otg19:30:031
1694994413,10cyclictest7850irq/56-dwc_otg_19:54:331
169699437,24cyclictest0-21swapper/323:53:023
1696994316,17cyclictest5123-21timerandwakeup23:48:153
1696994312,12cyclictest18786-21if_wlan021:28:083
169599438,23cyclictest0-21swapper/200:33:022
1694994318,11cyclictest14588-21/usr/sbin/munin19:48:051
1694994318,11cyclictest14588-21/usr/sbin/munin19:48:051
1694994314,12cyclictest27405-21taskset21:52:251
1694994314,12cyclictest27405-21taskset21:52:251
1694994313,14cyclictest10871-21taskset00:04:171
1694994310,11cyclictest9021-21chrt23:58:411
169699429,12cyclictest29518-21fschecks_count21:58:063
1696994215,16cyclictest25600-21sshd23:15:423
169599427,25cyclictest0-21swapper/222:48:022
169599427,25cyclictest0-21swapper/220:53:032
169599427,22cyclictest0-21swapper/221:43:032
169499427,25cyclictest0-21swapper/119:43:021
1694994215,12cyclictest0-21swapper/121:06:571
1694994214,13cyclictest1596-21taskset20:39:461
169699417,22cyclictest0-21swapper/322:08:023
169699417,22cyclictest0-21swapper/321:23:023
169699417,22cyclictest0-21swapper/319:28:033
169699417,21cyclictest0-21swapper/320:43:023
169599412,6cyclictest0-21swapper/219:33:012
169499412,28cyclictest0-21swapper/100:13:021
1694994119,13cyclictest27244-21seq20:22:061
1694994119,13cyclictest27244-21seq20:22:061
1694994117,11cyclictest19756-21taskset22:59:281
1694994115,17cyclictest23477-21runrttasks00:38:591
1694994114,12cyclictest12906-21run-parts22:41:071
1694994114,12cyclictest12906-21run-parts22:41:071
1694994111,11cyclictest23295-21chrt21:38:581
169699408,13cyclictest11278-21tune2fs19:38:063
169699408,11cyclictest28816-21munin-run21:55:043
169699408,11cyclictest28816-21munin-run21:55:043
169699407,23cyclictest11426-21fschecks_time21:08:083
169699407,21cyclictest0-21swapper/300:03:023
1696994013,17cyclictest9202-21apt19:33:013
169599406,24cyclictest0-21swapper/223:03:032
169599404,20cyclictest2491-21cut22:13:082
1695994021,5cyclictest1685950wget00:23:032
1695994011,7cyclictest31-21ksoftirqd/220:18:082
169499408,13cyclictest7950irq/56-dwc_otg_23:43:021
1694994017,11cyclictest5767-21taskset22:21:581
1694994014,18cyclictest7750irq/56-dwc_otg21:43:011
1694994011,13cyclictest494-21taskset23:37:111
1694994010,12cyclictest21674-21runrttasks20:05:111
169699392,26cyclictest0-21swapper/320:25:543
169599394,7cyclictest7850irq/56-dwc_otg_23:58:002
169599391,6cyclictest0-21swapper/219:28:002
1694993917,11cyclictest11298-21chrt00:06:421
1694993915,15cyclictest1124-21snmpd21:14:481
1694993911,9cyclictest0-21swapper/121:15:081
1694993911,10cyclictest7850irq/56-dwc_otg_22:13:551
169699381,6cyclictest12529-21unixbench_multi00:08:173
169599387,22cyclictest0-21swapper/222:18:022
169599381,9cyclictest8849-21idleruntime-cro21:00:032
1695993816,5cyclictest26098-21fschecks_time23:18:072
1694993814,5cyclictest16870-21cpuspeed22:53:041
169499381,11cyclictest14887-21aten_rbpower_cu22:48:031
169699377,16cyclictest20213-21taskset23:02:163
169699376,20cyclictest7850irq/56-dwc_otg_21:50:043
169699376,20cyclictest0-21swapper/323:41:513
169699376,18cyclictest7850irq/56-dwc_otg_22:13:203
169699375,4cyclictest7950irq/56-dwc_otg_20:48:173
1696993717,13cyclictest7850irq/56-dwc_otg_22:21:553
169699371,6cyclictest26678-21head20:18:183
1696993712,11cyclictest19322-21ntp_states00:28:123
169599371,4cyclictest7950irq/56-dwc_otg_20:23:012
169599371,4cyclictest7950irq/56-dwc_otg_20:23:012
169599371,25cyclictest3343-21ls23:43:162
169599371,21cyclictest7534-21ls23:55:042
1694993712,12cyclictest27444-21chrt23:20:571
1694993712,12cyclictest27444-21chrt23:20:571
1694993712,12cyclictest0-21swapper/123:32:091
169699362,3cyclictest0-21swapper/321:33:003
1696993620,5cyclictest22224-21fschecks_time20:08:083
169699361,20cyclictest9155-21ls22:30:043
169599365,15cyclictest7850irq/56-dwc_otg_22:55:032
169599362,6cyclictest1590-21sed22:08:162
169599361,7cyclictest32577-21latency_hist22:05:032
1695993616,14cyclictest0-21swapper/219:10:052
169599361,3cyclictest0-21swapper/223:33:012
169599361,25cyclictest23970-21cut20:13:082
1695993610,11cyclictest7850irq/56-dwc_otg_19:48:042
1695993610,11cyclictest7850irq/56-dwc_otg_19:48:042
169699358,5cyclictest7850irq/56-dwc_otg_19:18:093
169699352,3cyclictest0-21swapper/319:23:013
1696993518,8cyclictest14694-21basename19:48:033
1696993518,8cyclictest14694-21basename19:48:033
169599359,13cyclictest7950irq/56-dwc_otg_23:48:172
169599352,9cyclictest10716-21sed00:03:162
169599352,10cyclictest0-21swapper/222:33:022
169599351,23cyclictest13660-21missed_timers22:43:102
169599351,23cyclictest13660-21missed_timers22:43:102
313712340,1sleep20-21swapper/220:33:072
169699347,4cyclictest7850irq/56-dwc_otg_23:33:043
169699345,17cyclictest18192-21grep22:55:033
169699341,5cyclictest8065-21cut23:58:053
169599348,12cyclictest7850irq/56-dwc_otg_21:58:052
169599346,7cyclictest7850irq/56-dwc_otg_21:50:032
169599346,5cyclictest7850irq/56-dwc_otg_19:58:172
169599341,25cyclictest566-21cut20:38:072
169699335,14cyclictest17752-21modprobe19:55:023
169699334,16cyclictest27166-21chrt20:21:043
169699334,16cyclictest27166-21chrt20:21:043
169699333,10cyclictest7950irq/56-dwc_otg_21:12:083
169699331,22cyclictest0-21swapper/323:38:033
1696993312,13cyclictest288282chrt23:23:413
1696993312,13cyclictest288282chrt23:23:413
169599337,15cyclictest0-21swapper/221:35:092
169599332,9cyclictest0-21swapper/200:38:122
169599332,8cyclictest11323-21df_inode21:08:062
169599332,5cyclictest7750irq/56-dwc_otg00:13:012
169599331,8cyclictest10476-21grep21:03:172
169699328,14cyclictest9467-21/usr/sbin/munin22:33:003
169699327,12cyclictest7950irq/56-dwc_otg_20:39:533
1696993210,13cyclictest1124-21snmpd22:19:383
169599328,4cyclictest7750irq/56-dwc_otg19:38:062
169599324,6cyclictest31-21ksoftirqd/219:43:152
169599322,7cyclictest28054-21/usr/sbin/munin23:23:082
169599322,7cyclictest28054-21/usr/sbin/munin23:23:082
169599322,7cyclictest0-21swapper/221:54:192
169599322,7cyclictest0-21swapper/221:54:192
169599322,6cyclictest23422-21latency_hist20:10:042
169599322,13cyclictest7950irq/56-dwc_otg_20:48:162
169599321,6cyclictest17742-21apt19:55:022
169599321,21cyclictest0-21swapper/200:19:252
169599321,21cyclictest0-21swapper/200:19:252
169599321,13cyclictest20371-21cut20:03:062
169699319,13cyclictest7850irq/56-dwc_otg_00:30:563
169599315,11cyclictest7950irq/56-dwc_otg_23:26:352
169599312,18cyclictest0-21swapper/223:06:202
169599311,22cyclictest1492-21timerandwakeup23:38:162
14952316,13sleep21458-21ls19:10:042
169699305,15cyclictest0-21swapper/322:48:033
169699305,11cyclictest23871-21aten_rbpower_cu21:43:033
169699302,18cyclictest0-21swapper/319:41:153
169699301,19cyclictest0-21swapper/300:13:443
169699301,18cyclictest1124-21snmpd21:18:413
1696993010,10cyclictest381ktimersoftd/300:38:113
169599308,11cyclictest7950irq/56-dwc_otg_00:28:032
169599305,13cyclictest7950irq/56-dwc_otg_21:33:112
169599302,13cyclictest7950irq/56-dwc_otg_19:23:092
325022292,19sleep10-21swapper/119:07:451
169699296,12cyclictest8751-21taskset20:59:003
169699294,13cyclictest37-21rcuc/321:39:153
169699292,16cyclictest0-21swapper/320:33:063
169599298,5cyclictest25125-21grep23:13:172
169599297,5cyclictest7850irq/56-dwc_otg_21:28:022
169599294,4cyclictest7950irq/56-dwc_otg_22:58:052
169599294,3cyclictest7950irq/56-dwc_otg_21:18:082
169599291,25cyclictest2696-21missed_timers20:43:102
169599285,5cyclictest14448-21latency_hist21:15:042
169599282,3cyclictest7750irq/56-dwc_otg22:38:122
13422262,15sleep30-21swapper/319:09:123
1695992515,8cyclictest0-21swapper/219:18:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional