You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-13 - 23:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sat Dec 13, 2025 12:44:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
42806824320,21sleep30-21swapper/307:05:103
47439721140,4sleep00-21swapper/009:15:130
429428999079,7cyclictest0-21swapper/108:06:581
429428999079,7cyclictest0-21swapper/108:06:581
197399870,9rtkit-daemon1972-21rtkit-daemon07:35:372
197399860,8rtkit-daemon1972-21rtkit-daemon08:02:172
429420998372,6cyclictest0-21swapper/008:06:580
429420998372,6cyclictest0-21swapper/008:06:580
429428997820,53cyclictest2069-21gnome-shell09:33:591
42779527640,13sleep00-21swapper/007:05:040
429432997528,44cyclictest331rcuc/209:08:592
4856622720,5sleep0485660-21netstat09:45:230
42943299717,8cyclictest331rcuc/212:28:592
429428997012,35cyclictest534024-21taskset12:04:261
197399700,21rtkit-daemon1972-21rtkit-daemon07:19:172
197399700,21rtkit-daemon1972-21rtkit-daemon07:19:172
429435996962,4cyclictest4497-21PK-Backend08:06:583
429435996962,4cyclictest4497-21PK-Backend08:06:583
429428996958,7cyclictest2069-21gnome-shell07:11:581
42867626844,16sleep10-21swapper/107:05:211
429428996752,10cyclictest2069-21gnome-shell09:35:591
429428996511,11cyclictest0-21swapper/111:51:591
197399640,8rtkit-daemon1972-21rtkit-daemon10:05:403
429428996352,6cyclictest2069-21gnome-shell09:44:591
429428996352,6cyclictest2069-21gnome-shell09:44:591
429420996312,45cyclictest2069-21gnome-shell11:51:590
429420996311,9cyclictest2069-21gnome-shell08:53:580
429420996311,9cyclictest2069-21gnome-shell08:53:580
42898426344,13sleep20-21swapper/207:06:142
197399630,9rtkit-daemon1972-21rtkit-daemon08:35:582
42943299623,19cyclictest331rcuc/212:21:592
42942899629,11cyclictest0-21swapper/111:57:591
429428996251,6cyclictest2069-21gnome-shell09:27:591
429420996248,9cyclictest2069-21gnome-shell11:03:580
429420996212,46cyclictest2069-21gnome-shell11:57:590
197399620,8rtkit-daemon1972-21rtkit-daemon10:12:042
429428996114,15cyclictest475761-21taskset09:19:591
429428996110,47cyclictest0-21swapper/111:03:591
429420996111,8cyclictest2069-21gnome-shell12:08:590
429420996111,8cyclictest2069-21gnome-shell12:08:590
429420996110,8cyclictest2069-21gnome-shell11:17:590
42942899608,22cyclictest496512-21taskset10:16:051
429428996013,22cyclictest463293-40apt-check08:40:591
197399600,8rtkit-daemon1972-21rtkit-daemon12:24:363
4712292590,4sleep00-21swapper/009:05:150
42942899595,28cyclictest0-21swapper/111:00:011
42942899595,28cyclictest0-21swapper/111:00:011
42942899592,15cyclictest2069-21gnome-shell08:29:581
42942899592,15cyclictest2069-21gnome-shell08:29:581
42942099592,52cyclictest2069-21gnome-shell07:14:580
197399590,8rtkit-daemon1972-21rtkit-daemon11:41:273
197399590,8rtkit-daemon1972-21rtkit-daemon11:41:273
42942899588,31cyclictest466914-21taskset08:52:261
42942899588,31cyclictest466914-21taskset08:52:261
42942899588,19cyclictest438090-21timerwakeupswit07:30:301
42942899587,15cyclictest1081-21ntpd08:30:241
429428995814,19cyclictest502204-21taskset10:33:351
429428995814,19cyclictest502204-21taskset10:33:351
42943299572,14cyclictest2069-21gnome-shell07:52:582
42943299570,54cyclictest1972-21rtkit-daemon09:55:132
42942899578,27cyclictest541171-21taskset12:23:301
42942899572,51cyclictest2069-21gnome-shell07:18:581
42942899572,51cyclictest2069-21gnome-shell07:18:581
429428995720,10cyclictest0-21swapper/111:40:001
429428995714,19cyclictest538999-21taskset12:16:421
429428995713,20cyclictest443608-21taskset07:48:361
429428995713,16cyclictest535787-21taskset12:09:541
429428995713,16cyclictest535787-21taskset12:09:541
429428995713,15cyclictest470251-21taskset09:00:511
429428995712,21cyclictest477320-21taskset09:22:491
429428995712,15cyclictest525657-21taskset11:40:041
429428995712,15cyclictest525657-21taskset11:40:041
42942099574,14cyclictest1137-21runrttasks08:16:080
42943299562,50cyclictest2069-21gnome-shell10:36:592
42943299562,50cyclictest2069-21gnome-shell10:20:592
42942899568,19cyclictest442089-21latency_hist07:45:011
42942899568,19cyclictest442089-21latency_hist07:45:011
42942899567,15cyclictest522697-21kernelversion11:30:211
42942899562,50cyclictest2069-21gnome-shell08:58:581
429428995614,15cyclictest528523-21taskset11:45:351
429428995612,16cyclictest500462-21taskset10:28:281
429428995612,16cyclictest489901-21taskset09:59:591
42942099563,49cyclictest0-21swapper/007:17:580
42942099563,49cyclictest0-21swapper/007:17:580
429420995611,25cyclictest525168-21taskset11:36:540
197399560,9rtkit-daemon1972-21rtkit-daemon11:47:092
197399560,9rtkit-daemon1972-21rtkit-daemon11:03:102
197399560,9rtkit-daemon1972-21rtkit-daemon10:41:542
42943299553,11cyclictest455934-40lsb_release08:22:582
42943299552,49cyclictest2069-21gnome-shell10:50:592
42943299552,10cyclictest2069-21gnome-shell08:48:582
42943299551,50cyclictest2069-21gnome-shell11:30:592
42942899559,19cyclictest434218-21needreboot07:20:221
42942899559,16cyclictest461990-21date08:40:021
42942899558,16cyclictest451111-21cstates08:10:141
42942899557,18cyclictest517534-21/usr/sbin/munin11:15:261
429428995513,15cyclictest503823-21taskset10:37:091
42942099552,48cyclictest2069-21gnome-shell08:43:580
42942099552,10cyclictest2069-21gnome-shell11:33:590
42943299542,48cyclictest2069-21gnome-shell10:46:592
42942899547,19cyclictest545748-21grep12:35:221
42942899546,16cyclictest471860-21sed09:05:291
42942899542,48cyclictest2069-21gnome-shell12:27:591
429428995412,15cyclictest486343-21taskset09:48:441
429428995410,14cyclictest473214-21sed09:10:331
42942099543,48cyclictest0-21swapper/009:43:590
42942099543,48cyclictest0-21swapper/009:43:590
42942099542,48cyclictest2069-21gnome-shell12:23:590
42942099542,47cyclictest2069-21gnome-shell08:30:580
42942099541,18cyclictest507644-21sed10:50:020
42942099541,11cyclictest2069-21gnome-shell07:21:580
197399540,9rtkit-daemon1972-21rtkit-daemon12:39:302
197399540,9rtkit-daemon1972-21rtkit-daemon12:31:052
197399540,9rtkit-daemon1972-21rtkit-daemon12:31:042
42943599538,6cyclictest411rcuc/308:53:583
42943599538,6cyclictest411rcuc/308:53:583
42943299530,49cyclictest1972-21rtkit-daemon10:01:292
42943299530,49cyclictest1972-21rtkit-daemon08:44:552
42943299530,48cyclictest1972-21rtkit-daemon12:00:132
42942899539,17cyclictest505461-21taskset10:41:001
42942899538,16cyclictest2069-21gnome-shell12:30:091
42942899538,16cyclictest2069-21gnome-shell12:30:091
42942899534,8cyclictest231irq_work/110:20:591
42942899534,45cyclictest231irq_work/107:57:581
42942899531,47cyclictest2069-21gnome-shell12:11:581
429428995314,10cyclictest519198-21/usr/sbin/munin11:20:241
429428995313,16cyclictest521634-21taskset11:29:531
429420995340,10cyclictest0-21swapper/009:27:590
42942099532,8cyclictest0-21swapper/008:26:580
42942099532,8cyclictest0-21swapper/008:26:580
42942099531,48cyclictest2069-21gnome-shell09:38:590
42942099531,46cyclictest2069-21gnome-shell09:30:590
197399530,8rtkit-daemon1972-21rtkit-daemon07:27:162
429435995240,8cyclictest411rcuc/310:45:213
42942899528,18cyclictest490913-21ntp_kernel_pll_10:00:301
42942899524,45cyclictest231irq_work/108:46:581
429428995212,16cyclictest436658-21taskset07:28:561
429428995212,15cyclictest495063-21taskset10:14:401
42942099522,47cyclictest0-21swapper/012:39:590
42943599512,45cyclictest2069-21gnome-shell08:03:583
42943299511,46cyclictest2069-21gnome-shell07:48:582
42943299510,48cyclictest1972-21rtkit-daemon09:26:402
42942899519,16cyclictest516172-21taskset11:11:221
42942899517,20cyclictest439910-21runrttasks07:36:121
42942899517,16cyclictest507038-21timerandwakeup10:45:291
42942899512,47cyclictest231irq_work/108:03:581
429428995112,10cyclictest455586-21/usr/sbin/munin08:20:271
42942099512,45cyclictest0-21swapper/012:27:590
5164792500,13sleep3421ksoftirqd/311:15:003
429435995043,4cyclictest411rcuc/307:11:583
42943599501,42cyclictest4497-21PK-Backend08:26:583
42943599501,42cyclictest4497-21PK-Backend08:26:583
42943599501,38cyclictest0-21swapper/311:57:583
42942899508,17cyclictest488168-21taskset09:54:561
429428995016,29cyclictest453300-21grep08:15:221
42942899501,45cyclictest231irq_work/110:07:591
42942099508,16cyclictest501614-21taskset10:30:250
42942099508,16cyclictest501614-21taskset10:30:250
42942099505,18cyclictest443483-21taskset07:46:590
42942099502,7cyclictest0-21swapper/008:58:580
42942099502,7cyclictest0-21swapper/008:47:580
42942099502,45cyclictest2069-21gnome-shell08:14:580
42942099502,44cyclictest0-21swapper/010:29:590
42942099502,44cyclictest0-21swapper/010:24:590
42942099501,21cyclictest514940-21ls11:10:020
429435994942,5cyclictest411rcuc/312:39:593
42943599493,5cyclictest411rcuc/311:17:593
42943599492,44cyclictest4497-21PK-Backend11:46:593
429432994941,5cyclictest0-21swapper/211:11:592
42943299492,44cyclictest2069-21gnome-shell10:34:592
42943299492,44cyclictest2069-21gnome-shell10:34:592
429432994910,11cyclictest141rcu_preempt11:45:012
429432994910,11cyclictest141rcu_preempt11:45:002
42942899498,17cyclictest444901-21cat07:50:291
42942899497,18cyclictest514425-21runrttasks11:06:391
429420994940,5cyclictest0-21swapper/010:13:590
42942099493,43cyclictest0-21swapper/009:13:590
42942099492,7cyclictest0-21swapper/010:57:590
42942099492,7cyclictest0-21swapper/010:57:590
42942099492,7cyclictest0-21swapper/007:42:580
42942099492,7cyclictest0-21swapper/007:42:580
42942099492,6cyclictest0-21swapper/011:41:590
42942099492,6cyclictest0-21swapper/011:41:590
42942099492,6cyclictest0-21swapper/011:29:590
42942099492,6cyclictest0-21swapper/010:01:590
42942099492,44cyclictest0-21swapper/012:10:590
42942099492,44cyclictest0-21swapper/009:58:590
42942099492,44cyclictest0-21swapper/008:00:580
42942099492,44cyclictest0-21swapper/007:38:580
42942099492,43cyclictest0-21swapper/010:05:590
42942099492,43cyclictest0-21swapper/008:38:580
42942099492,24cyclictest0-21swapper/011:23:340
42942099492,18cyclictest495743-21/usr/sbin/munin10:15:160
429435994842,4cyclictest411rcuc/309:25:593
429435994837,7cyclictest514531-40lsb_release11:07:593
42943599482,6cyclictest4497-21PK-Backend10:17:593
42943599482,43cyclictest4497-21PK-Backend11:33:593
42943599482,42cyclictest4497-21PK-Backend10:32:593
42943599482,42cyclictest4497-21PK-Backend10:32:593
42943599482,37cyclictest4497-21PK-Backend09:45:593
42943599481,44cyclictest42-21ksoftirqd/310:02:593
42943599481,38cyclictest4497-21PK-Backend07:36:583
429432994836,8cyclictest2069-21gnome-shell07:57:582
42943299482,6cyclictest0-21swapper/209:13:592
42943299482,6cyclictest0-21swapper/207:20:582
42943299482,43cyclictest0-21swapper/209:36:592
42943299481,6cyclictest4497-21PK-Backend11:08:592
42943299480,44cyclictest1972-21rtkit-daemon10:25:352
42942099486,24cyclictest1137-21runrttasks10:40:520
429420994840,4cyclictest0-21swapper/008:23:580
429420994839,6cyclictest0-21swapper/009:20:590
429420994839,5cyclictest0-21swapper/012:02:590
42942099482,44cyclictest0-21swapper/011:11:590
42942099482,43cyclictest0-21swapper/012:17:590
42943599474,38cyclictest2069-21gnome-shell11:25:593
429435994741,4cyclictest411rcuc/311:03:583
42943599473,5cyclictest411rcuc/311:51:593
42943599472,6cyclictest4497-21PK-Backend09:14:593
42943599472,5cyclictest4497-21PK-Backend12:33:593
42943599472,5cyclictest4497-21PK-Backend12:33:593
42943599472,5cyclictest4497-21PK-Backend09:21:593
42943599472,42cyclictest4497-21PK-Backend08:38:583
42943599472,42cyclictest4497-21PK-Backend08:20:583
42943599472,42cyclictest4497-21PK-Backend08:18:583
42943599471,6cyclictest4497-21PK-Backend09:18:593
42943599471,42cyclictest4497-21PK-Backend09:04:593
42943599471,37cyclictest4497-21PK-Backend07:19:583
42943599471,37cyclictest4497-21PK-Backend07:19:583
42943299472,6cyclictest0-21swapper/208:56:582
42943299472,42cyclictest0-21swapper/211:19:592
429428994739,5cyclictest0-21swapper/110:51:591
42942099475,28cyclictest1137-21runrttasks07:31:580
429420994739,4cyclictest0-21swapper/007:51:580
42942099472,43cyclictest0-21swapper/007:57:580
42943599462,41cyclictest4497-21PK-Backend09:41:593
42943599462,41cyclictest4497-21PK-Backend09:41:593
42943599462,41cyclictest4497-21PK-Backend07:42:583
42943599462,41cyclictest4497-21PK-Backend07:42:583
42943599461,5cyclictest4497-21PK-Backend10:40:583
42943599461,5cyclictest4497-21PK-Backend07:21:583
42943599461,42cyclictest4497-21PK-Backend08:13:583
42943599461,42cyclictest4497-21PK-Backend07:56:583
42943599461,41cyclictest4497-21PK-Backend10:57:593
42943599461,41cyclictest4497-21PK-Backend10:57:593
42943599461,41cyclictest4497-21PK-Backend08:45:583
42943599461,37cyclictest4497-21PK-Backend12:11:593
429432994638,5cyclictest0-21swapper/209:02:592
42943299462,5cyclictest0-21swapper/209:33:592
42943299460,43cyclictest1972-21rtkit-daemon11:36:482
42942099465,14cyclictest0-21swapper/011:45:180
429420994632,9cyclictest2069-21gnome-shell09:51:590
429435994538,4cyclictest4497-21PK-Backend10:26:593
429435994538,4cyclictest4497-21PK-Backend09:37:593
429435994538,4cyclictest4497-21PK-Backend08:42:583
429435994538,4cyclictest4497-21PK-Backend07:49:583
42943599452,40cyclictest4497-21PK-Backend12:27:593
42943599452,40cyclictest4497-21PK-Backend09:34:593
42943599451,5cyclictest4497-21PK-Backend12:03:593
42943599451,41cyclictest4497-21PK-Backend09:06:593
42943599451,41cyclictest4497-21PK-Backend08:58:583
42943599451,41cyclictest4497-21PK-Backend07:33:583
429432994539,4cyclictest0-21swapper/211:50:592
429432994538,5cyclictest0-21swapper/212:07:592
429432994538,5cyclictest0-21swapper/212:07:582
42943299452,26cyclictest34-21ksoftirqd/209:40:272
42943299452,26cyclictest34-21ksoftirqd/209:40:272
429420994538,4cyclictest0-21swapper/010:39:590
42943599445,21cyclictest42-21ksoftirqd/309:59:553
42943599445,21cyclictest42-21ksoftirqd/307:29:323
429435994436,5cyclictest4497-21PK-Backend10:35:593
429435994436,5cyclictest0-21swapper/310:14:593
429435994435,6cyclictest411rcuc/308:31:193
42943599443,18cyclictest518522-21grep11:20:133
42943299444,5cyclictest141rcu_preempt08:19:582
429432994438,4cyclictest0-21swapper/207:34:582
429432994433,7cyclictest5214102sleep211:26:592
42943299442,40cyclictest0-21swapper/208:33:582
429432994418,8cyclictest34-21ksoftirqd/208:05:132
429432994418,8cyclictest34-21ksoftirqd/208:05:132
42942099440,16cyclictest508161-21df_inode10:50:160
42943599436,20cyclictest42-21ksoftirqd/310:20:563
42943599435,20cyclictest42-21ksoftirqd/311:36:133
429435994335,5cyclictest4497-21PK-Backend07:54:583
429432994337,4cyclictest0-21swapper/211:59:592
42942099437,16cyclictest436726-21taskset07:29:480
5441752420,7sleep00-21swapper/012:30:270
5441752420,7sleep00-21swapper/012:30:270
429420994234,5cyclictest0-21swapper/009:04:590
429435994132,6cyclictest411rcuc/312:18:113
429432994135,4cyclictest0-21swapper/210:56:592
429432994135,4cyclictest0-21swapper/210:56:592
429432994135,4cyclictest0-21swapper/207:13:582
429432994134,5cyclictest0-21swapper/208:14:582
42943299403,12cyclictest34-21ksoftirqd/210:05:152
429432994030,6cyclictest34-21ksoftirqd/208:50:292
429432994030,6cyclictest34-21ksoftirqd/208:50:292
42943299401,22cyclictest486180-21seq09:47:002
42943299401,21cyclictest538723-21/usr/sbin/munin12:15:282
42943299401,18cyclictest476914-21/usr/sbin/munin09:20:272
42943599398,9cyclictest411rcuc/309:53:153
429432993933,4cyclictest0-21swapper/208:25:582
429432993933,4cyclictest0-21swapper/208:25:582
42943299391,21cyclictest488219-21seq09:55:012
429435993830,5cyclictest411rcuc/310:53:393
429432993826,8cyclictest413573-21kworker/2:111:24:592
42943299381,12cyclictest441659-21chrt07:40:422
42943299381,12cyclictest441659-21chrt07:40:422
429432993810,16cyclictest34-21ksoftirqd/209:18:462
42943299380,34cyclictest1972-21rtkit-daemon10:15:352
42943299371,15cyclictest536460-21taskset12:10:172
42943599365,16cyclictest42-21ksoftirqd/312:06:403
42943599365,16cyclictest42-21ksoftirqd/312:06:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional